Re: [OMPI users] Handling output of processes

2009-02-05 Thread jody
Hi Ralph Thanks - i downloaded and installed openmpi-1.4a1r20435 and now everything works as it should: --output-filename : all processes write their outputs to the correct files --xterm : all specified processes opened their xterms I started my application with --xterm as i wrote in

Re: [OMPI users] Handling output of processes

2009-02-03 Thread Ralph Castain
Hi Jody Well, the problem with both the output filename and the xterm option was that I wasn't passing them back to the remote daemons under the ssh launch environment. I should have that corrected now - things will hopefully work with any tarball of r20407 or above. Let me know... Ralph

Re: [OMPI users] Handling output of processes

2009-02-03 Thread Ralph Castain
Ah! I know the problem - forgot you are running under ssh, so the environment doesn't get passed. I'll have to find a way to pass the output filename to the backend nodes...should have it later today. On Feb 3, 2009, at 11:09 AM, jody wrote: Hi Ralph --output-filename It creates files,

Re: [OMPI users] Handling output of processes

2009-02-03 Thread jody
Hi Ralph >> >> --output-filename >> It creates files, but only for the local processes: >> [jody@localhost neander]$ mpirun -np 8 -hostfile testhosts >> --output-filename gnana ./MPITest >> ... output ... >> [jody@localhost neander]$ ls -l gna* >> -rw-r--r-- 1 jody morpho 549 2009-02-03 18:02 gna

Re: [OMPI users] Handling output of processes

2009-02-03 Thread Ralph Castain
On Feb 3, 2009, at 10:16 AM, jody wrote: Hi Ralph Thanks for the fixes and the "!". --xterm: The "!" works, but i still don't have any xterms from my remote nodes even with all my xhost+ and -x DISPLAY tricks explained below :( --output-filename It creates files, but only for the local proces

Re: [OMPI users] Handling output of processes

2009-02-03 Thread jody
Hi Ralph Thanks for the fixes and the "!". --xterm: The "!" works, but i still don't have any xterms from my remote nodes even with all my xhost+ and -x DISPLAY tricks explained below :( --output-filename It creates files, but only for the local processes: [jody@localhost neander]$ mpirun -np 8

Re: [OMPI users] Handling output of processes

2009-02-02 Thread Ralph Castain
On Feb 2, 2009, at 2:55 AM, jody wrote: Hi Ralph The new options are great stuff! Following your suggestion, i downloaded and installed http://www.open-mpi.org/nightly/trunk/openmpi-1.4a1r20392.tar.gz and tested the new options. (i have a simple cluster of 8 machines over tcp). Not everything

Re: [OMPI users] Handling output of processes

2009-02-02 Thread Ralph Castain
Okay, I have this fixed and the man page updated as of r20396. Thanks again for finding and reporting this bug! Ralph On Feb 2, 2009, at 5:55 AM, Ralph Castain wrote: Hmnmm...well, it shouldn't crash (so I'll have to fix that), but it should fail. The --report-pid option takes an argument,

Re: [OMPI users] Handling output of processes

2009-02-02 Thread Ralph Castain
Great feedback - thanks! I'll look into these and see what's going on. We had tested things on a couple of systems, but it sounds like maybe there are some system- specific issues we are encountering. Let me explore a little to see what might be happening. Ralph On Feb 2, 2009, at 2:55 AM

Re: [OMPI users] Handling output of processes

2009-02-02 Thread Ralph Castain
Hmnmm...well, it shouldn't crash (so I'll have to fix that), but it should fail. The --report-pid option takes an argument, which wasn't provided here. I'll check the man page to ensure it is up-to-date. What it should tell you is that --report-pid takes either a '-' to indicate that the pi

Re: [OMPI users] Handling output of processes

2009-02-02 Thread jody
Hi Ralph one more thing i noticed while trying out orte_iof again. The option --report-pid crashes mpirun: [jody@localhost neander]$ mpirun -report-pid -np 2 ./MPITest [localhost:31146] *** Process received signal *** [localhost:31146] Signal: Segmentation fault (11) [localhost:31146] Signal code:

Re: [OMPI users] Handling output of processes

2009-02-02 Thread jody
Hi Ralph The new options are great stuff! Following your suggestion, i downloaded and installed http://www.open-mpi.org/nightly/trunk/openmpi-1.4a1r20392.tar.gz and tested the new options. (i have a simple cluster of 8 machines over tcp). Not everything worked as specified, though: * timestamp-ou

Re: [OMPI users] Handling output of processes

2009-02-01 Thread Ralph Castain
I'm afraid we discovered a bug in optimized builds with r20392. Please use any tarball with r20394 or above. Sorry for the confusion Ralph On Feb 1, 2009, at 5:27 AM, Jeff Squyres wrote: On Jan 31, 2009, at 11:39 AM, Ralph Castain wrote: For anyone following this thread: I have completed

Re: [OMPI users] Handling output of processes

2009-02-01 Thread Jeff Squyres
On Jan 31, 2009, at 11:39 AM, Ralph Castain wrote: For anyone following this thread: I have completed the IOF options discussed below. Specifically, I have added the following: * a new "timestamp-output" option that timestamp's each line of output * a new "output-filename" option that redi

Re: [OMPI users] Handling output of processes

2009-01-31 Thread Ralph Castain
For anyone following this thread: I have completed the IOF options discussed below. Specifically, I have added the following: * a new "timestamp-output" option that timestamp's each line of output * a new "output-filename" option that redirects each proc's output to a separate rank-named f

Re: [OMPI users] Handling output of processes

2009-01-26 Thread jody
That's cool then - i have written a shellscript which automatically does the xhost stuff for all nodes in my hostfile :) On Mon, Jan 26, 2009 at 9:25 PM, Ralph Castain wrote: > > On Jan 26, 2009, at 1:20 PM, jody wrote: > >> Hi Brian >> >>> >>> I would rather not have mpirun doing an xhost comman

Re: [OMPI users] Handling output of processes

2009-01-26 Thread Ralph Castain
On Jan 26, 2009, at 1:20 PM, jody wrote: Hi Brian I would rather not have mpirun doing an xhost command - I think that is beyond our comfort zone. Frankly, if someone wants to do this, it is up to them to have things properly setup on their machine - as a rule, we don't mess with your

Re: [OMPI users] Handling output of processes

2009-01-26 Thread jody
Typo there: "xceren" stands for "screen" - sorry :) On Mon, Jan 26, 2009 at 9:20 PM, jody wrote: > Hi Brian > >> >> I would rather not have mpirun doing an xhost command - I think that is >> beyond our comfort zone. Frankly, if someone wants to do this, it is up to >> them to have things properly

Re: [OMPI users] Handling output of processes

2009-01-26 Thread jody
Hi Brian > > I would rather not have mpirun doing an xhost command - I think that is > beyond our comfort zone. Frankly, if someone wants to do this, it is up to > them to have things properly setup on their machine - as a rule, we don't > mess with your machine's configuration. Makes sys admins u

Re: [OMPI users] Handling output of processes

2009-01-26 Thread Ralph Castain
Hi Jody I would rather not have mpirun doing an xhost command - I think that is beyond our comfort zone. Frankly, if someone wants to do this, it is up to them to have things properly setup on their machine - as a rule, we don't mess with your machine's configuration. Makes sys admins ups

Re: [OMPI users] Handling output of processes

2009-01-26 Thread Ralph Castain
Yes. The --tag-output option will prepend [job,rank] (or stderr, whichever is appropriate) to each line. I don't insert a colon, though I suppose that would easily be done for grep purposes. I just finished implementing the --output-filename option that will split the output from each rank

Re: [OMPI users] Handling output of processes

2009-01-26 Thread Douglas Guptill
Hello Ralph: Please forgive if this has already been covered... Have you considered prefixing each line of output from each process with something like "process_number" and a colon? That is what IBM's poe does. Separating the output is then easy: cat file | grep 0: > file.0 cat file | grep

Re: [OMPI users] Handling output of processes

2009-01-26 Thread Allen Barnett
On Sun, 2009-01-25 at 05:20 -0700, Ralph Castain wrote: > 2. redirect output of specified processes to files using the provided > filename appended with ".rank". You can do this for all ranks, or a > specified subset of them. A filename extension including both the comm size and the rank is h

Re: [OMPI users] Handling output of processes

2009-01-26 Thread jody
Hi I have written some shell scripts which ease the output to an xterm for each processor for normal execution(run_sh.sh), gdb (run_gdb.sh), and valgrind (run_vg.sh). In order for the xterms to be shown on your machine, you have to set the DISPLAY variable on every host (if this is not done by ssh

Re: [OMPI users] Handling output of processes

2009-01-25 Thread Ralph Castain
For those of you following this thread: I have been impressed by the various methods used to grab the output from processes. Since this is clearly something of interest to a broad audience, I would like to try and make this easier to do by adding some options to mpirun. Coming in 1.3.1 will

Re: [OMPI users] Handling output of processes

2009-01-23 Thread Gijsbert Wiesenekker
jody wrote: Hi I have a small cluster consisting of 9 computers (8x2 CPUs, 1x4 CPUs). I would like to be able to observe the output of the processes separately during an mpirun. What i currently do is to apply the mpirun to a shell script which opens a xterm for each process, which then starts t

Re: [OMPI users] Handling output of processes

2009-01-23 Thread Allen Barnett
On Thu, 2009-01-22 at 06:33 -0700, Ralph Castain wrote: > If you need to do this with a prior releasewell, I'm afraid it > won't work. :-) As a quick hack for 1.2.x, I sometimes use this script to wrap my executable: --- #!/bin/sh #

Re: [OMPI users] Handling output of processes

2009-01-22 Thread Ralph Castain
Hmmmit works fine for me. Note that orte-iof must be executed from the same node as mpirun. Also, ensure that rank=0 is actually the MPI rank that is printing something! Otherwise, nothing will come out. It should also be noted that it does take time for you to execute orte- iof, and tha

Re: [OMPI users] Handling output of processes

2009-01-22 Thread jody
Hi I downloaded 1.3 the day its release was announced :) I now wanted to try it, so i pieced it together from orte-iof --help: - started an mpirun with 4 process - opened a new terminal - did ps to find the mpirun's pid - then: orte-iof --pid 8449 --rank 0 --stdout but nothing happens... So did i

Re: [OMPI users] Handling output of processes

2009-01-22 Thread Ralph Castain
If you are willing/able to upgrade to 1.3, you will find a new utility called "ompi-iof" (or "orte-iof") that allows you to capture the output from any specified rank or combination of ranks. Note that the output is copied to the tool, so it will also still arrive at the output of mpirun.