Hi Jeff
Comments after your text....
Jeff Squyres wrote 2/5/07:
Greetings Joe.
What we did was to make 2 sets of environment variables that you can
use to modify OMPI's internal path behavior:
1. OPAL_DESTDIR: If you want to use staged builds (a la RPM builds),
you can set the environment variable OPAL_DESTDIR. For example:
./configure --prefix=/opt/ompi
make install DESTDIR=/tmp
# at this point, OMPI is installed into /tmp/opt/ompi
setenv OPAL_DESTDIR /tmp
# put /tmp/opt/ompi in your path, then run ompi_info
ompi_info
Hence, OPAL_DESTDIR simply prefixes every internal path with the
destdir.
2. OPAL_PREFIX: If you want to wholly override where the OMPI
installation is, set OPAL_PREFIX to the new prefix value (quite
similar to the LAMHOME environment variable, but slightly better).
For example:
./configure --prefix=/opt/ompi
make install
mv /opt/ompi /home/ompi
setenv OPAL_PREFIX /home/ompi
ompi_info
This will re-base all of OMPI's internal directories to key off the
new prefix value. Similarly, OPAL_* environment variables exist for
all other GNU-standard directories (OPAL_BINDIR,
OPAL_INCLUEDIR, ...etc.). Note that the default values of these
variables are their GNU counterparts, so OMPI_BINDIR defaults to $
{prefix}/bin -- which requires resolving ${prefix} first (i.e.,
OPAL_PREFIX). Hence, setting OPAL_PREFIX will effectively change all
the others. We anticipate that most people will likely only ever use
OPAL_PREFIX.
I finally got a chance to try V1.2. I am having the same issue:
I installed with:
./configure --prefix=/usr/local/openmpi-1.2-1
make prefix=/usr/local/openmpi-1.2-1
make install prefix=/usr/local/openmpi-1.2-1
I can run with:
em64t6 <155> /usr/local/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
em64t6
em64t6 <156> cp -r /usr/local/openmpi-1.2-1 /tmp
em64t6 <157> /tmp/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
em64t6
Then I move the directory:
[root@em64t6 openmpi-1.2]# cd /usr/local/
[root@em64t6 local]# mv openmpi-1.2-1/ openmpiHIDE
Then I try to run:
em64t6 <158> /tmp/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
/tmp/openmpi-1.2-1/bin/mpirun: error while loading shared libraries:
libopen-rte.so.0: cannot open shared object file: No such file or directory
If I set the env:
em64t6 <160> setenv OPAL_PREFIX /tmp/openmpi-1.2-1
em64t6 <164> setenv OPAL_DESTDIR /tmp/openmpi-1.2-1/
em64t6 <166> set path=( /tmp/openmpi-1.2-1/bin $path)
em64t6 <168> setenv LD_LIBRARY_PATH /tmp/openmpi-1.2-1/lib
The error becomes:
em64t6 <169> /tmp/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
[em64t6:30103] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_init_stage1.c at line 214
--------------------------------------------------------------------------
Sorry! You were supposed to get help about:
orte_init:startup:internal-failure
from the file:
help-orte-runtime
But I couldn't find any file matching that name. Sorry!
--------------------------------------------------------------------------
[em64t6:30103] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_system_init.c at line 42
[em64t6:30103] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_init.c at line 52
--------------------------------------------------------------------------
Sorry! You were supposed to get help about:
orterun:init-failure
from the file:
help-orterun.txt
But I couldn't find any file matching that name. Sorry!
--------------------------------------------------------------------------
The output from ompi_info is:
em64t6 <170> ompi_info
Open MPI: 1.2
Open MPI SVN revision: r14027
Open RTE: 1.2
Open RTE SVN revision: r14027
OPAL: 1.2
OPAL SVN revision: r14027
Prefix: /usr/local/openmpi-1.2-1
Configured architecture: x86_64-unknown-linux-gnu
Configured by: root
Configured on: Tue Apr 10 09:39:28 PDT 2007
Configure host: em64t6
Built by: root
Built on: Tue Apr 10 09:51:27 PDT 2007
Built host: em64t6
C bindings: yes
C++ bindings: yes
Fortran77 bindings: yes (all)
Fortran90 bindings: no
Fortran90 bindings size: na
C compiler: gcc
C compiler absolute: /usr/bin/gcc
C++ compiler: g++
C++ compiler absolute: /usr/bin/g++
Fortran77 compiler: g77
Fortran77 compiler abs: /usr/bin/g77
Fortran90 compiler: none
Fortran90 compiler abs: none
C profiling: yes
C++ profiling: yes
Fortran77 profiling: yes
Fortran90 profiling: no
C++ exceptions: no
Thread support: posix (mpi: no, progress: no)
Internal debug support: no
MPI parameter check: runtime
Memory profiling support: no
Memory debugging support: no
libltdl support: yes
Heterogeneous support: yes
mpirun default --prefix: no
MCA backtrace: execinfo (MCA v1.0, API v1.0, Component v1.2)
MCA memory: ptmalloc2 (MCA v1.0, API v1.0, Component v1.2)
MCA timer: linux (MCA v1.0, API v1.0, Component v1.2)
So the Prefix is not getting changed. Can someone please
tell me what I am doing wrong? I searched the archives,
but haven't found anything related.
I would like to:
1. install into one directory
2. Move the files
3. Run from the new directory.
The above is important if we redistribute and clients want to
install in a different directory.
Thanks,
Joe
-----
We've been calling this work "the installdirs patch" internally. It
got finalized on a private branch literally just before we closed the
1.2 tree for release, so it's not going to make it into the final
v1.2 -- it was just too late in the release process. It's not even
on the OMPI trunk yet; we wanted to get OMPI v1.2 out before putting
this patch on the trunk (to keep the trunk and v1.2 "somewhat
similar"). We haven't yet discussed 1.2.1 as a group yet -- it's
possible that the installdirs patch will make it into there (but I
can't guarantee that).
That being said, I will be releasing OMPI v1.2ofed as part of the
OpenFabrics Enterprise Distribution (OFED -- the open source
InfiniBand stack) that will be the final community OMPI v1.2 release
plus the installdirs patch. We need this installdirs stuff in the
OFED installer for DESTDIR reasons.
I'd be happy to share the installdirs patch with anyone who wants it;
it just won't make it into OMPI v1.2.
On Feb 2, 2007, at 6:28 PM, Joe Griffin wrote:
Hello,
I can see that Patrick posted my same error on:
http://www.open-mpi.org/community/lists/devel/2006/12/1230.php
Can I please find out the status? I have a simple test case which
demonstrates the --prefix is not working on mpirun. I build into
/usr/local/openmpi-1.1.3
and it works. However, if I copy the files, remove the
/usr/local/openmpi-1.1.3/lib
and try to point to my new directory of /tmp/openmpi-1.1.3, it fails.
prompt <154> /usr/local/openmpi-1.1.3/bin/mpirun -n 1 --host em64t
hostname
em64t
prompt <155> cp -r /usr/local/openmpi-1.1.3 /tmp
prompt <156> /tmp/openmpi-1.1.3/bin/mpirun -n 1 --host em64t hostname
em64t
# As ROOT:
[root@em64t openmpi-1.1.3]# mv /usr/local/openmpi-1.1.3/lib
/usr/local/openmpi-1.1.3/LLL
prompt <157> /tmp/openmpi-1.1.3/bin/mpirun -n 1 --host em64t --prefix
/tmp/openmpi-1.1.3 hostname
[em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_init_stage1.c at line 214
----------------------------------------------------------------------
----
It looks like orte_init failed for some reason; your parallel
process is
likely to abort. There are many reasons that a parallel process can
fail during orte_init; some of which are due to configuration or
environment problems. This failure appears to be an internal failure;
here's some additional information (which may only be relevant to an
Open MPI developer):
orte_sds_base_select failed
--> Returned value -13 instead of ORTE_SUCCESS
----------------------------------------------------------------------
----
[em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_system_init.c at line 42
[em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_init.c at line 49
----------------------------------------------------------------------
----
Open RTE was unable to initialize properly. The error occured while
attempting to orte_init(). Returned value -13 instead of
ORTE_SUCCESS.
----------------------------------------------------------------------
----
The man page implies that --prefix would work. I know my example is
silly. The real case
involves going over a cluster, but the above is the simpliest case I
could create.
From the man page:
--prefix <dir>
Prefix directory that will be used to set the
PATH and
LD_LIBRARY_PATH on the remote node before invoking
Open MPI
or the target process. See the "Remote Execution"
section,
below.
BTW - I am trying to migrate from LAM to OpenMPI. The switch was
very painless. Thanks for the good work.
Regards,
Joe
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
