if just "./configure" was used - it can detect mxm only if it is installed in /usr/include/...
by default mxm is installed in /opt/mellanox/mxm/... I just checked with: "./configure" and it did not detect mxm which is installed in the system space "./configure --with-mxm" and it did not detect mxm "./configure --with-mxm=/opt/mellanox/mxm" and it did work as expected. Can you please send me your config.log to understand how it could happen? On Tue, May 26, 2015 at 8:40 PM, David Shrader <dshra...@lanl.gov> wrote: > Hello Mike, > > I'm still working on getting you my config.log, but I thought I would > chime in about that line 36. In my case, that code path is not executed > because with_mxm is empty (I don't use --with-mxm on the configure line > since libmxm.so is in system space and configure picks up on it > automatically). Thus, ompi_check_mxm_libdir never gets assigned which > results in just "-L" getting used on line 41. The same behavior could be > found by using '--with-mxm=yes'. > > Thanks, > David > > > On 05/26/2015 11:28 AM, Mike Dubman wrote: > > Thanks Jeff! > > but in this line: > > https://github.com/open-mpi/ompi/blob/master/config/ompi_check_mxm.m4#L36 > > ompi_check_mxm_libdir gets value if with_mxm was passed > > > > On Tue, May 26, 2015 at 6:59 PM, Jeff Squyres (jsquyres) < > jsquy...@cisco.com> wrote: > >> This line: >> >> >> https://github.com/open-mpi/ompi/blob/master/config/ompi_check_mxm.m4#L41 >> >> doesn't check to see if $ompi_check_mxm_libdir is empty. >> >> >> > On May 26, 2015, at 11:50 AM, Mike Dubman <mi...@dev.mellanox.co.il> >> wrote: >> > >> > David, >> > Could you please send me your config.log file? >> > >> > Looking into config/ompi_check_mxm.m4 macro I don`t understand how it >> could happen. >> > >> > Thanks a lot. >> > >> > On Tue, May 26, 2015 at 6:41 PM, Mike Dubman <mi...@dev.mellanox.co.il> >> wrote: >> > Hello David, >> > Thanks for info and patch - will fix ompi configure logic with your >> patch. >> > >> > mxm can be installed in the system and user spaces - both are valid and >> supported logic. >> > >> > M >> > >> > On Tue, May 26, 2015 at 5:50 PM, David Shrader <dshra...@lanl.gov> >> wrote: >> > Hello Mike, >> > >> > This particular instance of mxm was installed using rpms that were >> re-rolled by our admins. I'm not 100% sure where they got them (HPCx or >> somewhere else). I myself am not using HPCx. Is there any particular reason >> why mxm shouldn't be in system space? If there is, I'll share it with our >> admins and try to get the install location corrected. >> > >> > As for what is causing the extra -L, it does look like an empty >> variable is used without checking that it is empty in configure. Line >> 246117 in the configure script provided by the openmpi-1.8.5.tar.bz2 >> tarball has this: >> > >> > ompi_check_mxm_extra_libs="-L$ompi_check_mxm_libdir" >> > >> > By invoking configure with '/bin/sh -x ./configure ...' and changing >> PS4 to output line numbers, I saw that line 246117 was setting >> ompi_check_mxm_extra_libs to just "-L". It turns out that configure does >> this in three separate locations. I put a check around all three instances >> like this: >> > >> > if test ! -z "$ompi_check_mxm_extra_libs"; then >> > ompi_check_mxm_extra_libs="-L$ompi_check_mxm_libdir" >> > fi >> > >> > And the spurious '-L' disappeared from the linking commands and make >> completed fine. >> > >> > So, it looks like there are two solutions: move the install location of >> mxm to not be in system-space or modify configure. Which one would be the >> better one for me to pursue? >> > >> > Thanks, >> > David >> > >> > >> > On 05/23/2015 12:05 AM, Mike Dubman wrote: >> >> Hi, >> >> >> >> How mxm was installed? by copying? >> >> >> >> The rpm based installation places mxm into /opt/mellanox/mxm and not >> into /usr/lib64/libmxm.so. >> >> >> >> Do you use HPCx (pack of OMPI and MXM and FCA)? >> >> You can download HPCX, extract it anywhere and compile OMPI pointing >> to mxm location under HPCX. >> >> >> >> Also, HPCx contains rpms for mxm and fca. >> >> >> >> >> >> M >> >> >> >> On Sat, May 23, 2015 at 1:07 AM, David Shrader <dshra...@lanl.gov> >> wrote: >> >> Hello, >> >> >> >> I'm getting a spurious '-L' flag when I have mxm installed in >> system-space (/usr/lib64/libmxm.so) which is causing an error at link time >> during make: >> >> >> >> ...output snipped... >> >> /bin/sh ../../../../libtool --tag=CC --mode=link gcc -std=gnu99 -O3 >> -DNDEBUG -I/opt/panfs/include -finline-functions -fno-strict-aliasing >> -pthread -module -avoid-version -o libmca_mtl_mxm.la mtl_mxm.lo >> mtl_mxm_cancel.lo mtl_mxm_component.lo mtl_mxm_endpoint.lo mtl_mxm_probe.lo >> mtl_mxm_recv.lo mtl_mxm_send.lo -lmxm -L -lrt -lm -lutil >> >> libtool: link: require no space between `-L' and `-lrt' >> >> make[2]: *** [libmca_mtl_mxm.la] Error 1 >> >> make[2]: Leaving directory >> `/turquoise/usr/projects/hpctools/dshrader/hpcsoft/openmpi/1.8.5/openmpi-1.8.5/ompi/mca/mtl/mxm' >> >> make[1]: *** [all-recursive] Error 1 >> >> make[1]: Leaving directory >> `/turquoise/usr/projects/hpctools/dshrader/hpcsoft/openmpi/1.8.5/openmpi-1.8.5/ompi' >> >> make: *** [all-recursive] Error 1 >> >> >> >> If I I use --with-mxm=no, then this error doesn't occur (as expected >> as the mxm component isn't touched). Has anyone run in to this before? >> >> >> >> Here is my configure line: >> >> >> >> ./configure --disable-silent-rules >> --with-platform=contrib/platform/lanl/toss/optimized-panasas --prefix=... >> >> >> >> I wonder if there is an empty variable that should contain the >> directory libmxm is in somewhere in configure since no directory is passed >> to --with-mxm which is then paired with a "-L". I think I'll go through the >> configure script while waiting to see if anyone else has run in to this. >> >> >> >> Thank you for any and all help, >> >> David >> >> >> >> -- >> >> David Shrader >> >> HPC-3 High Performance Computer Systems >> >> Los Alamos National Lab >> >> Email: dshrader <at> lanl.gov >> >> >> >> _______________________________________________ >> >> users mailing list >> >> us...@open-mpi.org >> >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2015/05/26904.php >> >> >> >> >> >> >> >> -- >> >> >> >> Kind Regards, >> >> >> >> M. >> >> >> >> >> >> _______________________________________________ >> >> users mailing list >> >> >> >> us...@open-mpi.org >> >> >> >> Subscription: >> >> http://www.open-mpi.org/mailman/listinfo.cgi/users >> >> >> >> Link to this post: >> >> http://www.open-mpi.org/community/lists/users/2015/05/26905.php >> > >> > -- >> > David Shrader >> > HPC-3 High Performance Computer Systems >> > Los Alamos National Lab >> > Email: dshrader <at> >> > lanl.gov >> > >> > _______________________________________________ >> > users mailing list >> > us...@open-mpi.org >> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> > Link to this post: >> http://www.open-mpi.org/community/lists/users/2015/05/26936.php >> > >> > >> > >> > -- >> > >> > Kind Regards, >> > >> > M. >> > >> > >> > >> > -- >> > >> > Kind Regards, >> > >> > M. >> > _______________________________________________ >> > users mailing list >> > us...@open-mpi.org >> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> > Link to this post: >> http://www.open-mpi.org/community/lists/users/2015/05/26938.php >> >> >> -- >> Jeff Squyres >> jsquy...@cisco.com >> For corporate legal information go to: >> http://www.cisco.com/web/about/doing_business/legal/cri/ >> >> _______________________________________________ >> users mailing list >> us...@open-mpi.org >> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users >> Link to this post: >> http://www.open-mpi.org/community/lists/users/2015/05/26940.php >> > > > > -- > > Kind Regards, > > M. > > > _______________________________________________ > users mailing listus...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/05/26941.php > > > -- > David Shrader > HPC-3 High Performance Computer Systems > Los Alamos National Lab > Email: dshrader <at> lanl.gov > > > _______________________________________________ > users mailing list > us...@open-mpi.org > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users > Link to this post: > http://www.open-mpi.org/community/lists/users/2015/05/26942.php > -- Kind Regards, M.
