Hi Rolf
I apologize for the scarce documentation - we are working on it, but
have a ways to go. I've tried to address your questions below. Please
feel free to ask more!
Ralph
Rolf Vandevaart wrote:
Greetings:
I am new to the Open MPI world, and I have been trying to get a better
understanding of the ORTE environment. At this point, I have a few
questions that I was hoping someone could answer.
1. I have heard mention of running the ORTE daemons in persistent mode,
however, I can find no details of how to do this. Are there arguments
to either orted or mpirun to make this work right?
Normally, we start a persistent daemon with:
orted --seed --persistent --scope=public
This will start the daemon and "daemonize" it so it keeps running until
told to die. The arguments worth noting are:
(a) --persistent. Tells the daemon to "stay alive" until specifically
told to "die"
(b) --scope=[public, private, exclusive]. This actually pertains to the
universe, but you'll need to provide it anyway to ensure proper
connectivity to anything you try to run. Right now, the daemons default
to "exclusive", which means nothing can connect to them except the
application that spawned them - no value to anyone if started with the
above command! Private would exclude them to contact only from you - I
haven't tested this enough to guarantee its functionality. I usually run
them as "public" since security isn't a big concern right now - all this
means is that anyone who can read the session directory tree (which is
normally "locked" to only you anyway) would be able to connect to the
daemon.
(c) --seed. Indicates that this daemon is the first one and therefore
will host the data storage for the registry and other central services
(d) --universe=userid@hostname:universe_name. Allows you to name your
universe to whatever you like. We use this to allow you to have multiple
universes co-existing but separate - I've been explaining the reasons
for that elsewhere, but will send them to this list if desired. You
don't have to provide this, nor do you have to provide all the fields
(e.g., you could just say "--universe=foo" to set the universe name).
You can provide the same options to mpirun, if you like - mpirun will
simply start an orted and pass those parameters along, and the orted
will merrily stay alive after the specified application completes.
2. I stumbled into a binary called orteconsole. Is this a useful
utility? I have played with it, but have found no documentation
on it so I am wondering what the state of it is.
I happen to like this utility myself - it allows you to connect to a
running universe (persistent or not - you can use it, for example, to
connect to the universe of a running application and see what is going
on) and explore the OpenRTE internal data structures, issue commands,
etc. Not everything is implemented yet - our initial need was just a way
of politely telling persistent daemons to "die" and cleanup after
themselves. I've forgotten which commands have been implemented, but can
look at it and write a "man" page for it if you like.
3. I have a similar question about orteprobe. Is this something
we should know about?
Yes and no - there's nothing secret about it. We use it internally to
OpenRTE to "probe" a machine and see if we have a daemon/universe
operating on it. Basically, we launch orteprobe on the remote machine -
it checks to see if a session directory exists on it, attempts to
connect to any universes it finds, and then reports back on its
findings. Based on that report, we either launch an orted on the remote
machine (to act as our surrogate so we can launch an application on that
cell) or connect to an existing universe on the remote machine (and then
tell it to launch the application for us).
4. Is there an easy way to view the data in the General Purpose
Registry? This may be related to my first question, in that I
could imagine having persistent daemons and then I would like
to see what is stored in the registry.
Well, yes and no. Ideally, that would be a command from within the
orteconsole function, but I don't think that has been implemented yet.
I'd be happy to do so, if that is something you would like (shouldn't
take long at all). There are a set of "dump" functions in the registry
API for just that purpose. I usually access them via gdb - I attach the
debugger to the orted process, then use the dump functions to output the
values in the registry.
Not as easy as the orteconsole interface would be, I admit.
5. Is there a way to monitor what processes are running? For
example, if I am running 3 MPI programs can I run some command
that would tell me this?
Josh has been working on an "orte_ps" command, but I don't think he has
it done yet.
6. From what I can tell, there is no way to specify the slots argument
with the -host argument. For example, I cannot do this:
mpirun -np 8 -host node1:slots=4,node2:slots=4 a.out
Just wanted to confirm that.
Now that's an interesting question! Since Jeff was the one who wrote all
that "hostfile" stuff, I'll have to defer to him - quick glance at the
code would seem to support your understanding, but I might have missed
something.
Thanks for any information,
Rolf
