Hi Erin
You should follow Dominik's and Reuti's suggestions,
and use the resource manager [Torque, Slurm, SGE]
built-in support for OpenMPI [and MPICH2 if you want]
Which resource manager is installed in your Rocks cluster,
depends on how it was built.
Rocks can be built with either SGE or Torque, and maybe not
so easily with Slurm as well.
You may need to ask the system administrator or
whoever built/knows the cluster.
However, 'man qsub' may give you a hint [will show PBS if you
have Torque/PBS and probably SGE if you have it.].
We have Torque here, so my answers are focused on Torque/PBS,
but there are equivalent workarounds for SGE, I guess.
***
My recollection is that the OpenMPI that comes native with
Rocks is *not* built with either SGE or Torque support.
Hence, it won't pick up the nodes' file that the resource manager
allocated to your job and use it as a machinefile, which is
what you probably want.
***
If you're using Torque, a workaround
with the native Rocks OpenMPI
is to use the $PBS_NODEFILE file as your machine file,
e.g., inside your job submission script:
cd $PBS_O_WORKDIR # this is to get to the work directory
mpiexec -np 32 -hostfile $PBS_NODEFILE ./my_mpi_program
***
A notch up, is to install an alternative built of
OpenMPI in your area, ensuring Torque or SGE support.
This is as easy as 'configure;make;install', as long as
you use the right flags to configure:
Download the source code:
http://www.open-mpi.org/software/ompi/v1.6/
You can use gcc,g++,gfotran, to build OpenMPI, if installed
in your cluster, as in the example below, or other compilers.
$ cd $HOME/Downloads
$ tar -jxvf openmpi-1.6.tar.bz2
$ ./configure --prefix=$HOME/openmpi-1.6.0 CC=gcc CXX=g++ F77=gfortran
FC=gfortran
If you have Torque, add this option to the command line above,
to get native Torque support:
--with-tm=/path/to/torque # wherever libtorque is installed
There is a similar option to build with SGE support, if you
have SGE, just do ./configure --help to see all options.
Also, if you have infiniband, and if it is installed in a
non-standard location, to build with infiniband support you
need to add this other option to the configure command line:
--with-openib=/path/to/openib # wherever librdma and libverbs are
installed
Then do:
$ make
$ make install
**
Check the README file to see if there were recent changes in
the configure options, please.
Do ./configure --help to see all options.
The FAQ are the actual OpenMPI documentation:
http://www.open-mpi.org/faq/
I hope this helps,
Gus Correa
On 07/04/2012 06:10 PM, Hodgess, Erin wrote:
Dear MPI people:
Is there a way (a script) available to automatically generate a
machinefile, please?
This would be on Rocks.
ompi_info -v ompi full --parsable
package:Open MPI r...@vi-1.rocksclusters.org Distribution
ompi:version:full:1.3.2
ompi:version:svn:r21054
ompi:version:release_date:Apr 21, 2009
orte:version:full:1.3.2
orte:version:svn:r21054
orte:version:release_date:Apr 21, 2009
opal:version:full:1.3.2
opal:version:svn:r21054
opal:version:release_date:Apr 21, 2009
ident:1.3.2
Thanks,
Erin
Erin M. Hodgess, PhD
Associate Professor
Department of Computer and Mathematical Sciences
University of Houston - Downtown
mailto: hodge...@uhd.edu
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
