Sorry, I'll try and fill in the background. I'm attempting to package
openmpi for a number of customers we have, whenever possible on our
clusters we use modules to provide users with a choice of MPI
environment.
I'm using the 1.2.6 stable release and have built the code twice, once
to /opt/openmpi-1.2.6/gnu and once to /opt/openmpi-1.2.6/intel, I have
create two modules environments called openmpi-gnu and openmpi-
intel and
am also using a existing one called intel-compiler. The build was
successful in both cases.
If I load the openmpi-gnu module I can compile and run code using
mpicc/mpirun as expected, if I load openmpi-intel and intel-compiler I
find I can compile code but I get an error about missing libimf.so
when
I try to run it (reproduced below).
The application *will* run if I add the line "module load
intel-compiler" to my bashrc as this allows orted to link. What I
think
I want to do is to compile the actual library with icc but to compile
orted with gcc so that I don't need to load the intel environment by
default. I'm assuming that the link problems only exist with orted
and
not with the actual application as the LD_LIBRARY_PATH is set
correctly
in the shell which is launching the program.
Ashley Pittman.
sccomp@demo4-sles-10-1-fe:~/benchmarks/IMB_3.0/src> mpirun -H
comp00,comp01 ./IMB-MPI1
/opt/openmpi-1.2.6/intel/bin/orted: error while loading shared
libraries: libimf.so: cannot open shared object file: No such file
or directory
/opt/openmpi-1.2.6/intel/bin/orted: error while loading shared
libraries: libimf.so: cannot open shared object file: No such file
or directory
[demo4-sles-10-1-fe:29303] ERROR: A daemon on node comp01 failed to
start as expected.
[demo4-sles-10-1-fe:29303] ERROR: There may be more information
available from
[demo4-sles-10-1-fe:29303] ERROR: the remote shell (see above).
[demo4-sles-10-1-fe:29303] ERROR: The daemon exited unexpectedly
with status 127.
[demo4-sles-10-1-fe:29303] [0,0,0] ORTE_ERROR_LOG: Timeout in file
base/pls_base_orted_cmds.c at line 275
[demo4-sles-10-1-fe:29303] [0,0,0] ORTE_ERROR_LOG: Timeout in file
pls_rsh_module.c at line 1166
[demo4-sles-10-1-fe:29303] [0,0,0] ORTE_ERROR_LOG: Timeout in file
errmgr_hnp.c at line 90
[demo4-sles-10-1-fe:29303] ERROR: A daemon on node comp00 failed to
start as expected.
[demo4-sles-10-1-fe:29303] ERROR: There may be more information
available from
[demo4-sles-10-1-fe:29303] ERROR: the remote shell (see above).
[demo4-sles-10-1-fe:29303] ERROR: The daemon exited unexpectedly
with status 127.
[demo4-sles-10-1-fe:29303] [0,0,0] ORTE_ERROR_LOG: Timeout in file
base/pls_base_orted_cmds.c at line 188
[demo4-sles-10-1-fe:29303] [0,0,0] ORTE_ERROR_LOG: Timeout in file
pls_rsh_module.c at line 1198
----------------------------------------------------------------------
----
mpirun was unable to cleanly terminate the daemons for this job.
Returned value Timeout instead of ORTE_SUCCESS.
----------------------------------------------------------------------
----
$ ldd /opt/openmpi-1.2.6/intel/bin/orted
linux-vdso.so.1 => (0x00007fff877fe000)
libopen-rte.so.0 => /opt/openmpi-1.2.6/intel/lib/libopen-
rte.so.0 (0x00007fe97f3ac000)
libopen-pal.so.0 => /opt/openmpi-1.2.6/intel/lib/libopen-
pal.so.0 (0x00007fe97f239000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007fe97f135000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00007fe97f01f000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007fe97ef1c000)
libm.so.6 => /lib64/libm.so.6 (0x00007fe97edc7000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007fe97ecba000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fe97eba3000)
libc.so.6 => /lib64/libc.so.6 (0x00007fe97e972000)
libimf.so => /opt/intel/compiler_10.1/x86_64/lib/libimf.so
(0x00007fe97e610000)
libsvml.so => /opt/intel/compiler_10.1/x86_64/lib/
libsvml.so (0x00007fe97e489000)
libintlc.so.5 => /opt/intel/compiler_10.1/x86_64/lib/
libintlc.so.5 (0x00007fe97e350000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe97f525000)
$ ssh comp00 ldd /opt/openmpi-1.2.6/intel/bin/orted
libopen-rte.so.0 => /opt/openmpi-1.2.6/intel/lib/libopen-
rte.so.0 (0x00002b1f0c0c5000)
libopen-pal.so.0 => /opt/openmpi-1.2.6/intel/lib/libopen-
pal.so.0 (0x00002b1f0c23e000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002b1f0c3bc000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00002b1f0c4c0000)
libutil.so.1 => /lib64/libutil.so.1 (0x00002b1f0c5d7000)
libm.so.6 => /lib64/libm.so.6 (0x00002b1f0c6da000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b1f0c82f000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b1f0c93d000)
libc.so.6 => /lib64/libc.so.6 (0x00002b1f0ca54000)
/lib64/ld-linux-x86-64.so.2 (0x00002b1f0bfa9000)
libimf.so => not found
libsvml.so => not found
libintlc.so.5 => not found
libimf.so => not found
libsvml.so => not found
libintlc.so.5 => not found
$ ldd ./IMB-MPI1
linux-vdso.so.1 => (0x00007fff2cbfe000)
libmpi.so.0 => /opt/openmpi-1.2.6/intel/lib/libmpi.so.0
(0x00007f1624821000)
libopen-rte.so.0 => /opt/openmpi-1.2.6/intel/lib/libopen-
rte.so.0 (0x00007f16246a8000)
libopen-pal.so.0 => /opt/openmpi-1.2.6/intel/lib/libopen-
pal.so.0 (0x00007f1624535000)
libdl.so.2 => /lib64/libdl.so.2 (0x00007f1624431000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f162431b000)
libutil.so.1 => /lib64/libutil.so.1 (0x00007f1624218000)
libm.so.6 => /lib64/libm.so.6 (0x00007f16240c3000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f1623fb6000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f1623e9f000)
libc.so.6 => /lib64/libc.so.6 (0x00007f1623c6e000)
libimf.so => /opt/intel/compiler_10.1/x86_64/lib/libimf.so
(0x00007f162390c000)
libsvml.so => /opt/intel/compiler_10.1/x86_64/lib/
libsvml.so (0x00007f1623785000)
libintlc.so.5 => /opt/intel/compiler_10.1/x86_64/lib/
libintlc.so.5 (0x00007f162364c000)
/lib64/ld-linux-x86-64.so.2 (0x00007f16249e0000)
On Mon, 2008-06-09 at 13:02 -0700, Doug Reeder wrote:
Ashley,
I am confused. In your first post you said orted fails, with link
errors, when you try to launch a job. From this I inferred that the
build and install steps for creating openmpi were successful. Was the
build/install step successful. If so what dynamic libraries does ldd
say that orted is using.
Doug Reeder
On Jun 9, 2008, at 12:54 PM, Ashley Pittman wrote:
Putting to side any religious views I might have about static
linking
how would that help in this case? It appears to be orted itself
that
fails to link, I'm assuming that the application would actually run,
either because the LD_LIBRARY_PATH is set correctly on the front
end or
the --prefix option to mpirun.
Or do you mean static linking of the tools? I could go for that if
there is a configure option for it.
Ashley Pittman.
On Mon, 2008-06-09 at 08:27 -0700, Doug Reeder wrote:
Ashley,
It could work but I think you would be better off to try and
statically link the intel libraries.
Doug Reeder
On Jun 9, 2008, at 4:34 AM, Ashley Pittman wrote:
Is there a way to use a different compiler for the orte component
and
the shared library component when using openmpi? We are finding
that if
we use icc to compile openmpi then orted fails with link errors
when I
try and launch a job as the intel environment isn't loaded by
default.
We use the module command heavily and have modules for openmpi-
gnu and
openmpi-intel as well as a intel_compiler module. To use openmpi-
intel
we have to load intel_compiler by default on the compute nodes
which
isn't ideal, is it possible to compile the orte component with
gcc and
the library component with icc?
Yours,
Ashley Pittman,
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
