We have been happily running under OpenMPI 1.2 on our cluster unitil
recently. It is 2200 processors (8 way Opteron) , Qlogic IB connected.
We have had issues starting larger jobs (600+ processors). There seemed
to be some indication that OpenMPI may solve our problems.
It built with no problem and installed. Users can compile programs.
When they tried to run, they got the attached output. Are we missing
something obvious?
This is a Rocks cluster with jobs scheduled through SGE.
=====================================================
$ mpirun -np 1024 program
[compute-2-6.local:32580] Error: unknown option "--daemonize"
Usage: orted [OPTION]...
Start an Open RTE Daemon
--bootproxy <arg0> Run as boot proxy for <job-id>
-d|--debug Debug the OpenRTE
-d|--spin Have the orted spin until we can connect a debugger
to it
--debug-daemons Enable debugging of OpenRTE daemons
--debug-daemons-file Enable debugging of OpenRTE daemons, storing
output
in files
--gprreplica <arg0> Registry contact information.
-h|--help This help message
--mpi-call-yield <arg0>
Have MPI (or similar) applications call yield when
idle
--name <arg0> Set the orte process name
--no-daemonize Don't daemonize into the background
--nodename <arg0> Node name as specified by host/resource
description.
--ns-nds <arg0> set sds/nds component to use for daemon (normally
not needed)
--nsreplica <arg0> Name service contact information.
--num_procs <arg0> Set the number of process in this job
--persistent Remain alive after the application process
completes
--report-uri <arg0> Report this process' uri on indicated pipe
--scope <arg0> Set restrictions on who can connect to this
universe
--seed Host replicas for the core universe services
--set-sid Direct the orted to separate from the current
session
--tmpdir <arg0> Set the root for the session directory tree
--universe <arg0> Set the universe name as
username@hostname:universe_name for this
application
--vpid_start <arg0> Set the starting vpid for this job
--------------------------------------------------------------------------
A daemon (pid 4151) died unexpectedly with status 251 while attempting
to launch so we are aborting.
There may be more information reported by the environment (see above).
This may be because the daemon was unable to find all the needed shared
libraries on the remote node. You may set your LD_LIBRARY_PATH to have the
location of the shared libraries on the remote nodes and this will
automatically be forwarded to the remote nodes.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun noticed that the job aborted, but has no info as to the process
that caused that situation.
--------------------------------------------------------------------------
--------------------------------------------------------------------------
mpirun was unable to cleanly terminate the daemons on the nodes shown
below. Additional manual cleanup may be required - please refer to
the "orte-clean" tool for assistance.
--------------------------------------------------------------------------
compute-5-15.local - daemon did not report back when launched
compute-5-35.local - daemon did not report back when launched
compute-4-8.local - daemon did not report back when launched
compute-7-2.local - daemon did not report back when launched
compute-2-6.local - daemon did not report back when launched
compute-6-28.local - daemon did not report back when launched
compute-6-35.local - daemon did not report back when launched
compute-6-25.local
compute-6-26.local
compute-2-19.local - daemon did not report back when launched
compute-6-37.local - daemon did not report back when launched
compute-6-12.local - daemon did not report back when launched
compute-2-36.local - daemon did not report back when launched
compute-7-5.local - daemon did not report back when launched
compute-7-23.local - daemon did not report back when launched
================================================
--
Ray Muno
University of Minnesota
