Hello Mike,
I'm glad that I could be of help.
Just as an FYI, right now our admins are still hosting the fca libraries
in /opt, but they would like to have it in system-space just as they
have done with mxm. I haven't worked my way through all of the
fca-related logic in configure yet, so I don't know if putting the fca
libraries in system-space will expose any issues as of yet. It might be
a good idea to check out that logic while looking at the mxm-related logic.
Thank you again!
David
On 05/26/2015 09:41 AM, Mike Dubman wrote:
Hello David,
Thanks for info and patch - will fix ompi configure logic with your patch.
mxm can be installed in the system and user spaces - both are valid
and supported logic.
M
On Tue, May 26, 2015 at 5:50 PM, David Shrader <dshra...@lanl.gov
<mailto:dshra...@lanl.gov>> wrote:
Hello Mike,
This particular instance of mxm was installed using rpms that were
re-rolled by our admins. I'm not 100% sure where they got them
(HPCx or somewhere else). I myself am not using HPCx. Is there any
particular reason why mxm shouldn't be in system space? If there
is, I'll share it with our admins and try to get the install
location corrected.
As for what is causing the extra -L, it does look like an empty
variable is used without checking that it is empty in configure.
Line 246117 in the configure script provided by the
openmpi-1.8.5.tar.bz2 tarball has this:
ompi_check_mxm_extra_libs="-L$ompi_check_mxm_libdir"
By invoking configure with '/bin/sh -x ./configure ...' and
changing PS4 to output line numbers, I saw that line 246117 was
setting ompi_check_mxm_extra_libs to just "-L". It turns out that
configure does this in three separate locations. I put a check
around all three instances like this:
if test ! -z "$ompi_check_mxm_extra_libs"; then
ompi_check_mxm_extra_libs="-L$ompi_check_mxm_libdir"
fi
And the spurious '-L' disappeared from the linking commands and
make completed fine.
So, it looks like there are two solutions: move the install
location of mxm to not be in system-space or modify configure.
Which one would be the better one for me to pursue?
Thanks,
David
On 05/23/2015 12:05 AM, Mike Dubman wrote:
Hi,
How mxm was installed? by copying?
The rpm based installation places mxm into /opt/mellanox/mxm and
not into /usr/lib64/libmxm.so.
Do you use HPCx (pack of OMPI and MXM and FCA)?
You can download HPCX, extract it anywhere and compile OMPI
pointing to mxm location under HPCX.
Also, HPCx contains rpms for mxm and fca.
M
On Sat, May 23, 2015 at 1:07 AM, David Shrader <dshra...@lanl.gov
<mailto:dshra...@lanl.gov>> wrote:
Hello,
I'm getting a spurious '-L' flag when I have mxm installed in
system-space (/usr/lib64/libmxm.so) which is causing an error
at link time during make:
...output snipped...
/bin/sh ../../../../libtool --tag=CC --mode=link gcc
-std=gnu99 -O3 -DNDEBUG -I/opt/panfs/include
-finline-functions -fno-strict-aliasing -pthread -module
-avoid-version -o libmca_mtl_mxm.la
<http://libmca_mtl_mxm.la> mtl_mxm.lo mtl_mxm_cancel.lo
mtl_mxm_component.lo mtl_mxm_endpoint.lo mtl_mxm_probe.lo
mtl_mxm_recv.lo mtl_mxm_send.lo -lmxm -L -lrt -lm -lutil
libtool: link: require no space between `-L' and `-lrt'
make[2]: *** [libmca_mtl_mxm.la <http://libmca_mtl_mxm.la>]
Error 1
make[2]: Leaving directory
`/turquoise/usr/projects/hpctools/dshrader/hpcsoft/openmpi/1.8.5/openmpi-1.8.5/ompi/mca/mtl/mxm'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory
`/turquoise/usr/projects/hpctools/dshrader/hpcsoft/openmpi/1.8.5/openmpi-1.8.5/ompi'
make: *** [all-recursive] Error 1
If I I use --with-mxm=no, then this error doesn't occur (as
expected as the mxm component isn't touched). Has anyone run
in to this before?
Here is my configure line:
./configure --disable-silent-rules
--with-platform=contrib/platform/lanl/toss/optimized-panasas
--prefix=...
I wonder if there is an empty variable that should contain
the directory libmxm is in somewhere in configure since no
directory is passed to --with-mxm which is then paired with a
"-L". I think I'll go through the configure script while
waiting to see if anyone else has run in to this.
Thank you for any and all help,
David
--
David Shrader
HPC-3 High Performance Computer Systems
Los Alamos National Lab
Email: dshrader <at> lanl.gov <http://lanl.gov>
_______________________________________________
users mailing list
us...@open-mpi.org <mailto:us...@open-mpi.org>
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2015/05/26904.php
--
Kind Regards,
M.
_______________________________________________ users mailing
list us...@open-mpi.org <mailto:us...@open-mpi.org> Subscription:
http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this
post:http://www.open-mpi.org/community/lists/users/2015/05/26905.php
--
David Shrader
HPC-3 High Performance Computer Systems
Los Alamos National Lab
Email: dshrader <at>lanl.gov <http://lanl.gov>
_______________________________________________
users mailing list
us...@open-mpi.org <mailto:us...@open-mpi.org>
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2015/05/26936.php
--
Kind Regards,
M.
_______________________________________________
users mailing list
us...@open-mpi.org
Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
http://www.open-mpi.org/community/lists/users/2015/05/26937.php
--
David Shrader
HPC-3 High Performance Computer Systems
Los Alamos National Lab
Email: dshrader <at> lanl.gov
