please redownload from http://mellanox.com/downloads/hpc/mxm/v1.1/mxm-latest.tar it contains binaries compiled with mofed 1.5.3-3.1.0 M
On Sun, Dec 2, 2012 at 12:13 PM, Mike Dubman <mike.o...@gmail.com> wrote: > > It seems that your active mofed is 1.5.3-3.1.0, while installed mxm was > compiled with 1.5.3-3.0.0 > MOFED is not binary compatible, let me check and send you the link for mxm > compiled with mofed that you have. > > Also, MOFED contains ompi 1.6.0 which is already compiled with mxm > (/usr/mpi/...) > On Sun, Dec 2, 2012 at 12:01 PM, Joseph Farran <jfar...@uci.edu> wrote: > >> Same thing. >> >> My new config: >> >> CFLAGS="" FCFLAGS="" ./configure \ >> --with-sge \ >> --with-openib=/usr \ >> >> --enable-openib-connectx-xrc \ >> --enable-mpi-thread-multiple \ >> --with-threads \ >> --with-hwloc \ >> --enable-heterogeneous \ >> --with-fca=/opt/mellanox/fca \ >> --with-mxm-libdir=/opt/mellanox/mxm/lib \ >> --with-mxm=/opt/mellanox/mxm \ >> >> Fails at the same spot: >> >> >> make[2]: Entering directory >> `/data/apps/sources/openmpi-1.6.3/ompi/mca/mtl/mxm' >> CC mtl_mxm.lo >> CC mtl_mxm_cancel.lo >> CC mtl_mxm_endpoint.lo >> CC mtl_mxm_probe.lo >> CC mtl_mxm_recv.lo >> >> CCLD mca_mtl_mxm.la >> /bin/grep: /usr/local/mofed-inst/1.5.3-3.0.0/lib/librdmacm.la: No such >> file or directory >> /bin/sed: can't read /usr/local/mofed-inst/1.5.3-3.0.0/lib/librdmacm.la: >> No such file or directory >> libtool: link: `/usr/local/mofed-inst/1.5.3-3.0.0/lib/librdmacm.la' is >> not a valid libtool archive >> make[2]: *** [mca_mtl_mxm.la] Error 1 >> make[2]: Leaving directory >> `/data/apps/sources/openmpi-1.6.3/ompi/mca/mtl/mxm' >> make[1]: *** [all-recursive] Error 1 >> make[1]: Leaving directory `/data/apps/sources/openmpi-1.6.3/ompi' >> make: *** [all-recursive] Error 1 >> >> >> >> On 12/2/2012 1:37 AM, Mike Dubman wrote: >> >> please change "--with-openib" to "--with-openib=/usr" and retry >> configure/make stage. >> 10x >> >> On Sun, Dec 2, 2012 at 10:36 AM, Joseph Farran <jfar...@uci.edu> wrote: >> >>> Hi Mike. >>> >>> Thanks for the help! >>> >>> I am installing OFED on an NFS share partition so that all compute nodes >>> will have access. >>> >>> For the "--with-openib" option, I don't specify one. My config file >>> looks like this: >>> >>> CFLAGS="" FCFLAGS="" ./configure \ >>> >>> --with-sge \ >>> --with-openib \ >>> --enable-openib-connectx-xrc \ >>> --enable-mpi-thread-multiple \ >>> --with-threads \ >>> --with-hwloc \ >>> --enable-heterogeneous \ >>> --with-fca=/opt/mellanox/fca \ >>> --with-mxm-libdir=/opt/mellanox/mxm/lib \ >>> --with-mxm=/opt/mellanox/mxm \ >>> --prefix=/data/openmpi-1-6.3 >>> >>> Please advise, >>> Joseph >>> >>> >>> >>> >>> >>> >>> On 12/1/2012 11:39 PM, Mike Dubman wrote: >>> >>> Hi Joseph, >>> I guess you install MOFED under /usr, is that right? >>> Could you please specify "--with-openib=/usr" parameter during ompi >>> "configure" stage? >>> 10x >>> M >>> >>> On Fri, Nov 30, 2012 at 1:11 AM, Joseph Farran <jfar...@uci.edu> wrote: >>> >>>> Hi YK: >>>> >>>> Yes, I have those installed but they are newer versions: >>>> >>>> # rpm -qa | grep rdma >>>> librdmacm-1.0.15-1.x86_64 >>>> librdmacm-utils-1.0.15-1.x86_64 >>>> librdmacm-devel-1.0.15-1.x86_64 >>>> # locate librdmacm.la >>>> # >>>> >>>> Here are the RPMs the Mellanox build created for kernel: >>>> 2.6.32-279.14.1.el6.x86_64 >>>> >>>> # ls *rdma* >>>> librdmacm-1.0.15-1.i686.rpm librdmacm-devel-1.0.15-1.i686.rpm >>>> librdmacm-utils-1.0.15-1.i686.rpm >>>> librdmacm-1.0.15-1.x86_64.rpm librdmacm-devel-1.0.15-1.x86_64.rpm >>>> librdmacm-utils-1.0.15-1.x86_64.rpm >>>> >>>> >>>> On 11/29/2012 02:59 PM, Yevgeny Kliteynik wrote: >>>> >>>>> Joseph, >>>>> >>>>> >>>>> You're supposed to have librdmacm installed as part of MLNX_OFED >>>>> installation. >>>>> What does "rpm -qa | grep rdma" tell? >>>>> >>>>> $ rpm -qa | grep rdma >>>>> librdmacm-devel-1.0.14.1-1.x86_64 >>>>> librdmacm-utils-1.0.14.1-1.x86_64 >>>>> librdmacm-1.0.14.1-1.x86_64 >>>>> >>>>> $ locate librdmacm.la >>>>> /usr/local/mofed/1.5.3-4.0.9/lib/librdmacm.la >>>>> >>>>> -- YK >>>>> >>>>> >>>> >>> >>> >> >> >
