It seems that your active mofed is 1.5.3-3.1.0, while installed mxm was compiled with 1.5.3-3.0.0 MOFED is not binary compatible, let me check and send you the link for mxm compiled with mofed that you have.
Also, MOFED contains ompi 1.6.0 which is already compiled with mxm (/usr/mpi/...) On Sun, Dec 2, 2012 at 12:01 PM, Joseph Farran <jfar...@uci.edu> wrote: > Same thing. > > My new config: > > CFLAGS="" FCFLAGS="" ./configure \ > --with-sge \ > --with-openib=/usr \ > > --enable-openib-connectx-xrc \ > --enable-mpi-thread-multiple \ > --with-threads \ > --with-hwloc \ > --enable-heterogeneous \ > --with-fca=/opt/mellanox/fca \ > --with-mxm-libdir=/opt/mellanox/mxm/lib \ > --with-mxm=/opt/mellanox/mxm \ > > Fails at the same spot: > > > make[2]: Entering directory > `/data/apps/sources/openmpi-1.6.3/ompi/mca/mtl/mxm' > CC mtl_mxm.lo > CC mtl_mxm_cancel.lo > CC mtl_mxm_endpoint.lo > CC mtl_mxm_probe.lo > CC mtl_mxm_recv.lo > > CCLD mca_mtl_mxm.la > /bin/grep: /usr/local/mofed-inst/1.5.3-3.0.0/lib/librdmacm.la: No such > file or directory > /bin/sed: can't read /usr/local/mofed-inst/1.5.3-3.0.0/lib/librdmacm.la: > No such file or directory > libtool: link: `/usr/local/mofed-inst/1.5.3-3.0.0/lib/librdmacm.la' is > not a valid libtool archive > make[2]: *** [mca_mtl_mxm.la] Error 1 > make[2]: Leaving directory > `/data/apps/sources/openmpi-1.6.3/ompi/mca/mtl/mxm' > make[1]: *** [all-recursive] Error 1 > make[1]: Leaving directory `/data/apps/sources/openmpi-1.6.3/ompi' > make: *** [all-recursive] Error 1 > > > > On 12/2/2012 1:37 AM, Mike Dubman wrote: > > please change "--with-openib" to "--with-openib=/usr" and retry > configure/make stage. > 10x > > On Sun, Dec 2, 2012 at 10:36 AM, Joseph Farran <jfar...@uci.edu> wrote: > >> Hi Mike. >> >> Thanks for the help! >> >> I am installing OFED on an NFS share partition so that all compute nodes >> will have access. >> >> For the "--with-openib" option, I don't specify one. My config file >> looks like this: >> >> CFLAGS="" FCFLAGS="" ./configure \ >> >> --with-sge \ >> --with-openib \ >> --enable-openib-connectx-xrc \ >> --enable-mpi-thread-multiple \ >> --with-threads \ >> --with-hwloc \ >> --enable-heterogeneous \ >> --with-fca=/opt/mellanox/fca \ >> --with-mxm-libdir=/opt/mellanox/mxm/lib \ >> --with-mxm=/opt/mellanox/mxm \ >> --prefix=/data/openmpi-1-6.3 >> >> Please advise, >> Joseph >> >> >> >> >> >> >> On 12/1/2012 11:39 PM, Mike Dubman wrote: >> >> Hi Joseph, >> I guess you install MOFED under /usr, is that right? >> Could you please specify "--with-openib=/usr" parameter during ompi >> "configure" stage? >> 10x >> M >> >> On Fri, Nov 30, 2012 at 1:11 AM, Joseph Farran <jfar...@uci.edu> wrote: >> >>> Hi YK: >>> >>> Yes, I have those installed but they are newer versions: >>> >>> # rpm -qa | grep rdma >>> librdmacm-1.0.15-1.x86_64 >>> librdmacm-utils-1.0.15-1.x86_64 >>> librdmacm-devel-1.0.15-1.x86_64 >>> # locate librdmacm.la >>> # >>> >>> Here are the RPMs the Mellanox build created for kernel: >>> 2.6.32-279.14.1.el6.x86_64 >>> >>> # ls *rdma* >>> librdmacm-1.0.15-1.i686.rpm librdmacm-devel-1.0.15-1.i686.rpm >>> librdmacm-utils-1.0.15-1.i686.rpm >>> librdmacm-1.0.15-1.x86_64.rpm librdmacm-devel-1.0.15-1.x86_64.rpm >>> librdmacm-utils-1.0.15-1.x86_64.rpm >>> >>> >>> On 11/29/2012 02:59 PM, Yevgeny Kliteynik wrote: >>> >>>> Joseph, >>>> >>>> >>>> You're supposed to have librdmacm installed as part of MLNX_OFED >>>> installation. >>>> What does "rpm -qa | grep rdma" tell? >>>> >>>> $ rpm -qa | grep rdma >>>> librdmacm-devel-1.0.14.1-1.x86_64 >>>> librdmacm-utils-1.0.14.1-1.x86_64 >>>> librdmacm-1.0.14.1-1.x86_64 >>>> >>>> $ locate librdmacm.la >>>> /usr/local/mofed/1.5.3-4.0.9/lib/librdmacm.la >>>> >>>> -- YK >>>> >>>> >>> >> >> > >
