Dear OpenMPI experts
I had some trouble to build OpenMPI 1.3.1 on a CentOS 5.2 Linux x86_64
computer.
The configure scripts seem to have changed, and work different
than before, particularly w.r.t. additional libraries like numa,
torque, and openib.
The new behavior can be a bit unexpected and puzzled me,
although eventually I could build 1.3.1.
Here are my observations.
1) I used to configure OpenMPI 1.2.8 and 1.3.0 with:
--with-libnuma=/usr/lib64 \
--with-tm=/usr/lib64 \
--with-openib=/usr/lib64 \
This worked fine for me on the same computer I am using for 1.3.1.
However, with 1.3.1 the same options fail.
Configure now tries to find the corresponding include
files on /usr/lib64/include, a directory that doesn't even exist.
The include files are actually in /usr/include
(as the old configure knew well).
2) Therefore, I tried to configure with:
--with-libnuma \
--with-tm \
--with-openib \
Note that no directory is being pointed to.
My hope was that configure would find the libraries and includes in
standard places (and hopefully the correct libs, 64-bit, not 32-bit).
This way configure completes with no apparent error.
However, I get this funny error on the make phase:
/bin/sh ../../../../libtool --tag=CC --mode=link gcc -DNDEBUG
-march=amdfam10 -O3 -finline-functions -funroll-loops -mfpmath=sse
-fno-strict-aliasing -pthread -fvisibility=hidden -module -avoid-version
-Lyes/lib -export-dynamic -o libmca_maffinity_libnuma.la
maffinity_libnuma_component.lo maffinity_libnuma_module.lo -lnuma -lnsl
-lutil -lm
../../../../libtool: line 4998: cd: yes/lib: No such file or directory
libtool: link: cannot determine absolute directory name of `yes/lib'
make[2]: *** [libmca_maffinity_libnuma.la] Error 1
Note the "yes/lib" path.
A little grep on config.log showed why the error:
%grep yes config.log
...
OMPI_WRAPPER_EXTRA_LDFLAGS=' -Lyes/lib '
OPAL_WRAPPER_EXTRA_LDFLAGS='-Lyes/lib '
ORTE_WRAPPER_EXTRA_LDFLAGS=' -Lyes/lib '
WRAPPER_EXTRA_LDFLAGS=' -Lyes/lib '
maffinity_libnuma_CPPFLAGS=' -Iyes/include'
maffinity_libnuma_LDFLAGS=' -Lyes/lib'
#define WRAPPER_EXTRA_LDFLAGS " -Lyes/lib "
Is this an internal "yes" answer to configure that
is being inadvertently caught/interpreted as a directory name?
Since configure seems to have found the libraries and include files,
and completed without error,
shouldn't it also have reported the correct paths to config.log
and written them correctly to the Makefiles?
3) Finally I tried this:
--with-libnuma=/usr \
--with-tm=/usr \
--with-openib=/usr \
This approach was suggested by Prentice Bisbal a few days ago,
when Francesco Pietra reported on this list
having a similar problem with libnuma.
This seems to work fine, and OpenMPI 1.3.1 builds.
However, I have more questions:
A)Is the directory name mandatory or optional in the options above?
I.e. is "--with-libnuma" only OK, or do I have to use
"--with-libnuma=/some/directory"?
The results in 2) above suggest that configure finds the libraries and
includes correctly, but that it writes wrong Makefiles,
and doesn't report any error either.
B) Is the syntax in 3) above the only correct possibility?
C) If it is, can I rest assured that configure and make
will find the right 64-bit libraries, not 32-bit libraries
of similar name?
I ask because I have /usr/lib/libnuma.so.1 (32-bit ELF),
and /usr/lib64/libnuma.so.1 (64-bit ELF), and both are in the
same /usr directory that I gave to configure (--with-libnuma=/usr).
(Well, maybe this is deferred to the compiler to decide,
whether it is a 64- or 32-bit compiler, as somehow it seemed to work.)
Thank you,
Gus Correa
---------------------------------------------------------------------
Gustavo Correa
Lamont-Doherty Earth Observatory - Columbia University
Palisades, NY, 10964-8000 - USA
---------------------------------------------------------------------
