[OMPI users] About the necessity of cancelation of pending communication and the use of buffer

2010-05-25 Thread Yves Caniou
Dear All,

I have 2 questions on the use of the MPI API:

1 ** Do I have to cancel Irecv() and Isend() operations before calling 
MPI_Finalize() in the case of when MPI_Finalize() is called, I know for sure 
that the application has finished to compute the result, and must exit?

A silly example would be an iterative code that Isends() a value given a 
proba, and checks if it receives a value given another proba. At the end, 
each prints a value. There's no synchronization. Do I have to track all 
Isend() and cancel them before calling MPI_Finalize()?

2 ** When I use a Isend() operation, the manpage says that I can't use the 
buffer until the operation completes.
What happens if I use an Isend() operation in a function, with a buffer 
declared inside the function?
Do I have to Wait() for the communication to finish before returning, or to 
declare the buffer as a global variable?

Thank you!

.Yves.

-- 
Yves Caniou
Associate Professor at Université Lyon 1,
Member of the team project INRIA GRAAL in the LIP ENS-Lyon,
Délégation CNRS in Japan French Laboratory of Informatics (JFLI),
  * in Information Technology Center, The University of Tokyo,
2-11-16 Yayoi, Bunkyo-ku, Tokyo 113-8658, Japan
tel: +81-3-5841-0540
  * in National Institute of Informatics
2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430, Japan
tel: +81-3-4212-2412 
http://graal.ens-lyon.fr/~ycaniou/



Re: [OMPI users] [sge::tight-integration] slot scheduling and resources handling

2010-05-25 Thread Eloi Gaudry
Hi Reuti,

I do no reset any environment variable during job submission or job handling.
Is there a simple way to check that openmpi is working as expected with SGE 
tight integration (as displaying environment variables, setting options on the 
command line, etc. ) ?

Regards,
Eloi


On Friday 21 May 2010 17:35:24 Reuti wrote:
> Hi,
> 
> Am 21.05.2010 um 17:19 schrieb Eloi Gaudry:
> > Hi Reuti,
> > 
> > Yes, the openmpi binaries used were build after having used the
> > --with-sge during configure, and we only use those binaries on our
> > cluster.
> > 
> > [eg@moe:~]$ /opt/openmpi-1.3.3/bin/ompi_info
> > 
> > MCA ras: gridengine (MCA v2.0, API v2.0, Component
> > v1.3.3)
> 
> ok. As you have a Tight Integration as goal and set in your PE
> "control_slaves TRUE", SGE wouldn't allow `qrsh -inherit ...` to nodes
> which are not in the list of granted nodes. So it looks, like your job is
> running outside of this Tight Integration with its own `rsh`or `ssh`.
> 
> Do you reset $JOB_ID or other environment variables in your jobscript,
> which could trigger Open MPI to assume that it's not running inside SGE?
> 
> -- Reuti
> 
> > On Friday 21 May 2010 16:01:54 Reuti wrote:
> >> Hi,
> >> 
> >> Am 21.05.2010 um 14:11 schrieb Eloi Gaudry:
> >>> Hi there,
> >>> 
> >>> I'm observing something strange on our cluster managed by SGE6.2u4 when
> >>> launching a parallel computation on several nodes, using OpenMPI/SGE
> >>> tight- integration mode (OpenMPI-1.3.3). It seems that the SGE
> >>> allocated slots are not used by OpenMPI, as if OpenMPI was doing is
> >>> own
> >>> round-robin allocation based on the allocated node hostnames.
> >> 
> >> you compiled Open MPI with --with-sge (and recompiled your
> >> applications)? You are using the correct mpiexec?
> >> 
> >> -- Reuti
> >> 
> >>> Here is what I'm doing:
> >>> - launch a parallel computation involving 8 processors, using for each
> >>> of them 14GB of memory. I'm using a qsub command where i request
> >>> memory_free resource and use tight integration with openmpi
> >>> - 3 servers are available:
> >>> . barney with 4 cores (4 slots) and 32GB
> >>> . carl with 4 cores (4 slots) and 32GB
> >>> . charlie with 8 cores (8 slots) and 64GB
> >>> 
> >>> Here is the output of the allocated nodes (OpenMPI output):
> >>> ==   ALLOCATED NODES   ==
> >>> 
> >>> Data for node: Name: charlie   Launch id: -1 Arch: ffc91200  State: 2
> >>> 
> >>> Daemon: [[44332,0],0] Daemon launched: True
> >>> Num slots: 4  Slots in use: 0
> >>> Num slots allocated: 4  Max slots: 0
> >>> Username on node: NULL
> >>> Num procs: 0  Next node_rank: 0
> >>> 
> >>> Data for node: Name: carl.fftLaunch id: -1 Arch: 0 State: 2
> >>> 
> >>> Daemon: Not defined Daemon launched: False
> >>> Num slots: 2  Slots in use: 0
> >>> Num slots allocated: 2  Max slots: 0
> >>> Username on node: NULL
> >>> Num procs: 0  Next node_rank: 0
> >>> 
> >>> Data for node: Name: barney.fftLaunch id: -1 Arch: 0 State: 2
> >>> 
> >>> Daemon: Not defined Daemon launched: False
> >>> Num slots: 2  Slots in use: 0
> >>> Num slots allocated: 2  Max slots: 0
> >>> Username on node: NULL
> >>> Num procs: 0  Next node_rank: 0
> >>> 
> >>> =
> >>> 
> >>> Here is what I see when my computation is running on the cluster:
> >>> # rank   pid  hostname
> >>> 
> >>>0 28112  charlie
> >>>1 11417  carl
> >>>2 11808  barney
> >>>3 28113  charlie
> >>>4 11418  carl
> >>>5 11809  barney
> >>>6 28114  charlie
> >>>7 11419  carl
> >>> 
> >>> Note that -the parallel environment used under SGE is defined as:
> >>> [eg@moe:~]$ qconf -sp round_robin
> >>> pe_nameround_robin
> >>> slots  32
> >>> user_lists NONE
> >>> xuser_listsNONE
> >>> start_proc_args/bin/true
> >>> stop_proc_args /bin/true
> >>> allocation_rule$round_robin
> >>> control_slaves TRUE
> >>> job_is_first_task  FALSE
> >>> urgency_slots  min
> >>> accounting_summary FALSE
> >>> 
> >>> I'm wondering why OpenMPI didn't use the allocated nodes chosen by SGE
> >>> (cf. "ALLOCATED NODES" report) but instead allocate each job of the
> >>> parallel computation at a time, using a round-robin method.
> >>> 
> >>> Note that I'm using the '--bynode' option in the orterun command line.
> >>> If the behavior I'm observing is simply the consequence of using this
> >>> option, please let me know. This would eventually mean that one need
> >>> to state that SGE tight- integration has a lower priority on orterun
> >>> behavior than the different command line options.
> >>> 
> >>> Any help would be appreciated,
> >>> Thanks,
> >>> Eloi
> >> 
> >> ___
> >> users mailing list
> >> us...

Re: [OMPI users] Process doesn't exit on remote machine when using hostfile

2010-05-25 Thread Shiqing Fan

Hi,

What's the firewall setting on the remote node? Could you try to add an 
exception for the application, or turn off the firewall completely?


Regards,
Shiqing



On 2010-5-24 4:44 PM, Rajnesh Jindel wrote:
When I specify the hosts separately on the commandline, as follows, 
the process completes as expected.

mpirun -np 8 -host remotehost,localhost myapp
Output appears for the localhost and a textfile is created on the 
remotehost


However when I use a hostfile the remote processes never complete. I 
can see the output from the local processes and by remote login I can 
see that that processes are being started on the remote machine but 
never complete.


The is a simple reduce example using boost.mpi (v1.43) I'm using 
windows 7 x64 pro on both machines and openmpi 1.4.2 the hostfile and 
athe app are in the same locaion on both machines.


Any idea why this is happening?

Raj


___
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users



--
--
Shiqing Fan  http://www.hlrs.de/people/fan
High Performance Computing   Tel.: +49 711 685 87234
  Center Stuttgart (HLRS)Fax.: +49 711 685 65832
Address:Allmandring 30   email: f...@hlrs.de
70569 Stuttgart



Re: [OMPI users] Building from the SRPM version creates an rpm with striped libraries

2010-05-25 Thread Ashley Pittman

This is a standard rpm feature although like most things it can be disabled.

According to this mail and it's replies the two %defines below will prevent 
striping and the building of debuginfo rpms.

http://lists.rpm.org/pipermail/rpm-list/2009-January/000122.html

%define debug_package %{nil}
%define __strip /bin/true

Ashley.

On 25 May 2010, at 00:25, Peter Thompson wrote:

> I have a user who prefers building rpm's from the srpm.  That's okay, but for 
> debugging via TotalView it creates a version with the openmpi .so files 
> stripped and we can't gain control of the processes when launched via mpirun 
> -tv.  I've verified this with my own build of a 1.4.1 rpm which I then 
> installed and noticed the same behavior that the user reports.  I was hoping 
> to give them some advice as to how to avoid the stripping, as it appears that 
> the actual build of those libraries is done with -g and everything looks 
> fine.  But I can't figure out in the build (from the log file I created) just 
> where that stripping takes place, or how to get around it if need be.  The 
> best guess I have is that it may be happening at the very end when an rpm-tmp 
> file is executed, but that file has disappeared so I don't really know what 
> it does.  I thought it might be apparent in the spec file, but it's certainly 
> not apparent to me!  Any help or advice would be appreciated.

-- 

Ashley Pittman, Bath, UK.

Padb - A parallel job inspection tool for cluster computing
http://padb.pittman.org.uk




Re: [OMPI users] [sge::tight-integration] slot scheduling and resources handling

2010-05-25 Thread Reuti
Hi,

Am 25.05.2010 um 09:14 schrieb Eloi Gaudry:

> I do no reset any environment variable during job submission or job handling.
> Is there a simple way to check that openmpi is working as expected with SGE 
> tight integration (as displaying environment variables, setting options on 
> the 
> command line, etc. ) ?

a) put a command:

env

in the jobscript and check the output for $JOB_ID and various $SGE_* variables.

b) to confirm the misbehavior: are the tasks on the slave nodes kids of 
sge_shepherd or any system sshd/rshd?

-- Reuti


> Regards,
> Eloi
> 
> 
> On Friday 21 May 2010 17:35:24 Reuti wrote:
>> Hi,
>> 
>> Am 21.05.2010 um 17:19 schrieb Eloi Gaudry:
>>> Hi Reuti,
>>> 
>>> Yes, the openmpi binaries used were build after having used the
>>> --with-sge during configure, and we only use those binaries on our
>>> cluster.
>>> 
>>> [eg@moe:~]$ /opt/openmpi-1.3.3/bin/ompi_info
>>> 
>>>MCA ras: gridengine (MCA v2.0, API v2.0, Component
>>>v1.3.3)
>> 
>> ok. As you have a Tight Integration as goal and set in your PE
>> "control_slaves TRUE", SGE wouldn't allow `qrsh -inherit ...` to nodes
>> which are not in the list of granted nodes. So it looks, like your job is
>> running outside of this Tight Integration with its own `rsh`or `ssh`.
>> 
>> Do you reset $JOB_ID or other environment variables in your jobscript,
>> which could trigger Open MPI to assume that it's not running inside SGE?
>> 
>> -- Reuti
>> 
>>> On Friday 21 May 2010 16:01:54 Reuti wrote:
 Hi,
 
 Am 21.05.2010 um 14:11 schrieb Eloi Gaudry:
> Hi there,
> 
> I'm observing something strange on our cluster managed by SGE6.2u4 when
> launching a parallel computation on several nodes, using OpenMPI/SGE
> tight- integration mode (OpenMPI-1.3.3). It seems that the SGE
> allocated slots are not used by OpenMPI, as if OpenMPI was doing is
> own
> round-robin allocation based on the allocated node hostnames.
 
 you compiled Open MPI with --with-sge (and recompiled your
 applications)? You are using the correct mpiexec?
 
 -- Reuti
 
> Here is what I'm doing:
> - launch a parallel computation involving 8 processors, using for each
> of them 14GB of memory. I'm using a qsub command where i request
> memory_free resource and use tight integration with openmpi
> - 3 servers are available:
> . barney with 4 cores (4 slots) and 32GB
> . carl with 4 cores (4 slots) and 32GB
> . charlie with 8 cores (8 slots) and 64GB
> 
> Here is the output of the allocated nodes (OpenMPI output):
> ==   ALLOCATED NODES   ==
> 
> Data for node: Name: charlie   Launch id: -1 Arch: ffc91200  State: 2
> 
> Daemon: [[44332,0],0] Daemon launched: True
> Num slots: 4  Slots in use: 0
> Num slots allocated: 4  Max slots: 0
> Username on node: NULL
> Num procs: 0  Next node_rank: 0
> 
> Data for node: Name: carl.fftLaunch id: -1 Arch: 0 State: 2
> 
> Daemon: Not defined Daemon launched: False
> Num slots: 2  Slots in use: 0
> Num slots allocated: 2  Max slots: 0
> Username on node: NULL
> Num procs: 0  Next node_rank: 0
> 
> Data for node: Name: barney.fftLaunch id: -1 Arch: 0 State: 2
> 
> Daemon: Not defined Daemon launched: False
> Num slots: 2  Slots in use: 0
> Num slots allocated: 2  Max slots: 0
> Username on node: NULL
> Num procs: 0  Next node_rank: 0
> 
> =
> 
> Here is what I see when my computation is running on the cluster:
> # rank   pid  hostname
> 
>   0 28112  charlie
>   1 11417  carl
>   2 11808  barney
>   3 28113  charlie
>   4 11418  carl
>   5 11809  barney
>   6 28114  charlie
>   7 11419  carl
> 
> Note that -the parallel environment used under SGE is defined as:
> [eg@moe:~]$ qconf -sp round_robin
> pe_nameround_robin
> slots  32
> user_lists NONE
> xuser_listsNONE
> start_proc_args/bin/true
> stop_proc_args /bin/true
> allocation_rule$round_robin
> control_slaves TRUE
> job_is_first_task  FALSE
> urgency_slots  min
> accounting_summary FALSE
> 
> I'm wondering why OpenMPI didn't use the allocated nodes chosen by SGE
> (cf. "ALLOCATED NODES" report) but instead allocate each job of the
> parallel computation at a time, using a round-robin method.
> 
> Note that I'm using the '--bynode' option in the orterun command line.
> If the behavior I'm observing is simply the consequence of using this
> option, please let me know. This would eventually mean that one need
>

Re: [OMPI users] Process doesn't exit on remote machine when using hostfile

2010-05-25 Thread Rajnesh Jindel
disabled the firewall and using admin account so security isnt the issue
here. like I said this problem only occurs when using a hostfile,  if I
actually specify the hostname on the commandfline it works fine
On 25 May 2010 09:08, Shiqing Fan  wrote:

>  Hi,
>
> What's the firewall setting on the remote node? Could you try to add an
> exception for the application, or turn off the firewall completely?
>
> Regards,
> Shiqing
>
>
>
>
> On 2010-5-24 4:44 PM, Rajnesh Jindel wrote:
>
> When I specify the hosts separately on the commandline, as follows, the
> process completes as expected.
> mpirun -np 8 -host remotehost,localhost myapp
>  Output appears for the localhost and a textfile is created on the
> remotehost
>
> However when I use a hostfile the remote processes never complete. I can
> see the output from the local processes and by remote login I can see that
> that processes are being started on the remote machine but never complete.
>
> The is a simple reduce example using boost.mpi (v1.43) I'm using windows 7
> x64 pro on both machines and openmpi 1.4.2 the hostfile and athe app are in
> the same locaion on both machines.
>
> Any idea why this is happening?
>
> Raj
>
>
> ___
> users mailing list
> users@open-mpi.orghttp://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
>
> --
> --
> Shiqing Fan  http://www.hlrs.de/people/fan
> High Performance Computing   Tel.: +49 711 685 87234
>   Center Stuttgart (HLRS)Fax.: +49 711 685 65832
> Address:Allmandring 30   email: f...@hlrs.de
> 70569 Stuttgart
>
>


Re: [OMPI users] About the necessity of cancelation of pending communication and the use of buffer

2010-05-25 Thread Fernando Lemos
On Tue, May 25, 2010 at 1:03 AM, Yves Caniou  wrote:
> 2 ** When I use a Isend() operation, the manpage says that I can't use the
> buffer until the operation completes.
> What happens if I use an Isend() operation in a function, with a buffer
> declared inside the function?
> Do I have to Wait() for the communication to finish before returning, or to
> declare the buffer as a global variable?

If you declare it inside the function (an auto variable), you're
declaring it on the stack. When the function is over, the stack may be
reused and this is gonna have nasty effects. You don't need to declare
the buffer as a global, just allocate it on the heap (with new or
malloc or whatever), just make sure you don't lose track of it cause
you're probably gonna need to free that memory eventually.


Re: [OMPI users] Some Questions on Building OMPI on Linux Em64t

2010-05-25 Thread Jeff Squyres
On May 24, 2010, at 2:02 PM, Michael E. Thomadakis wrote:

> | > 1) high-resolution timers: how do I specify the HRT linux timers in the
> | > --with-timer=TYPE
> | >  line of ./configure ?
> |
> | You shouldn't need to do anything; the "linux" timer component of Open MPI
> | should get automatically selected.  You should be able to see this in the
> | stdout of Open MPI's "configure", and/or if you run ompi_info | grep timer
> | -- there should only be one entry: linux.
> 
> If nothing is menioned, will it by default select 'linux' timers?

Yes.

> Or I have to specify in th configure
> 
> --with-timer=linux ?

Nope.  The philosophy of Open MPI is that whenever possible, we try to choose a 
sensible default.  It never hurts to double check, but we try to do the Right 
Thing whenever it's possible to automatically choose it (within reason, of 
course).

You can also check the output of ompi_info -- ompi_info tells you lots of 
things about your Open MPI installation.

> I actually spent some time looking around in the source trying to see which
> actual timer is the base. Is this a high-resolution timer such as a POSIX
> timers (timer_gettime or clock_nanosleep, etc.) or Intel processor's TSC ?
> 
> I am just trying to stay away from gettimeofday()

Understood.

Ugh; I just poked into the code -- it's complicated how we resolve the timer 
functions.  It looks like we put in the infrastructure into getting high 
resolution timers, but at least for Linux, we don't use it (the code falls back 
to gettimeofday).  It looks like we're only using the high-resolution timers on 
AIX (!) and Solaris.

Patches would be greatly appreciated; I'd be happy to walk someone through what 
to do. 

> | > 2) I have installed blcr V0.8.2 but when I try to built OMPI and I point 
> to the
> | > full installation it complains it cannot find it. Note that I build BLCR 
> with
> | > GCC but I am building OMPI with Intel compilers (V11.1)
> |
> | Can you be more specific here?
> 
> I pointed to the insatllation path for BLCR but config complained that it
> couldn't find it. If BLCR is only needed for checkpoint / restart then we can
> leave without it. Is BLCR needed for suspend/resume of mpi jobs ?

You mean suspend with ctrl-Z?  If so, correct -- BLCR is *only* used for 
checkpoint/restart.  Ctrl-Z just uses the SIGSTP functionality.

> | > 4) How could I select the high-speed transport, say DAPL or OFED IB verbs 
> ? Is
> | > there any preference as to the specific high-speed transport over QDR IB?
> |
> | openib is the preferred Open MPI plugin (the name is somewhat outdated, but
> | it's modern OpenFabrics verbs -- see
> | http://www.open-mpi.org/faq/?category=openfabrics#why-openib-name).
> 
> Does this mean that the DAPL API is not suported at all or that OPMI works
> better with OFED verbs?

Sorry for not being clear.

Our DAPL plugin is only supported on Solaris.  It *probably* works on Linux 
(the API should be the same), but we don't test it on Linux at all.  The OMPI 
configure script deactivates the udapl plugin on Linux by default.

On Linux, DAPL is a (thin) layer over verbs, anyway, so there isn't much point 
in using it.  On Linux, Open MPI uses the verb (openib) plugin.  FWIW, DAPL is 
an abstraction layer that intentionally hides lower-layer things.  The verbs 
API is much more complex than DAPL and exposes a lot more information, which 
OMPI uses.

We strongly recommend that you use the verbs (openib) plugin on Linux.

> Justr as a feedback from one of the many HPC centers, for us it is most
> important to have
> 
> a) a light-weight efficient MPI stack which makes the underlying IB h/w
> capabilities available and
> 
> b) it can smoothly cooperate withe a batch scheduler / resource manager so
> that a mixture of jobs get a decent allocation of the cluster resources.

Cools; good to know.  We try to make these things very workable in Open MPI -- 
it's been a goal from day 1 to integrate with job schedulers, etc.  And without 
high performance, we wouldn't have much to talk about.

Please be sure to let us know of questions / problems / etc.  I admit that 
we're sometimes a little slow to answer on the users list, but we do the best 
we can.  So don't hesitate to bump us if we don't reply.

Thanks!

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/




Re: [OMPI users] About the necessity of cancelation of pending communication and the use of buffer

2010-05-25 Thread Richard Treumann






Dick Treumann  -  MPI Team
IBM Systems & Technology Group
Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846 Fax (845) 433-8363


users-boun...@open-mpi.org wrote on 05/25/2010 12:03:11 AM:

> [image removed]
>
> [OMPI users] About the necessity of cancelation of pending
> communication and the use of buffer
>
> Yves Caniou
>
> to:
>
> Open MPI Users
>
> 05/25/2010 12:04 AM
>
> Sent by:
>
> users-boun...@open-mpi.org
>
> Please respond to yves.caniou, Open MPI Users
>
> Dear All,
>
> I have 2 questions on the use of the MPI API:
>
> 1 ** Do I have to cancel Irecv() and Isend() operations before calling
> MPI_Finalize() in the case of when MPI_Finalize() is called, I know for
sure
> that the application has finished to compute the result, and must exit?
>
> A silly example would be an iterative code that Isends() a value given a
> proba, and checks if it receives a value given another proba. At the end,

> each prints a value. There's no synchronization. Do I have to track all
> Isend() and cancel them before calling MPI_Finalize()?

The MPI Standard requires that all posted communications be locally
complete before a call to MPI_Finalize. That means every non blocking send
or receive must be MPI_WAITed (or successful TESTed).  If the matching
operation will never be posted because the program has done all it needs
to, then you must both MPI_CANCEL those operations and MPI_WAIT them.

This is what the standard requires but that does not mean a particular MPI
implementation will fail if you ignore the rule. If you want to write
portable MPI code, obey the rule.

>
> 2 ** When I use a Isend() operation, the manpage says that I can't use
the
> buffer until the operation completes.
> What happens if I use an Isend() operation in a function, with a buffer
> declared inside the function?
> Do I have to Wait() for the communication to finish before returning, or
to
> declare the buffer as a global variable?
I see somebody answered this part.
>
> Thank you!
>
> .Yves.
>
> --
> Yves Caniou
> Associate Professor at Université Lyon 1,
> Member of the team project INRIA GRAAL in the LIP ENS-Lyon,
> Délégation CNRS in Japan French Laboratory of Informatics (JFLI),
>   * in Information Technology Center, The University of Tokyo,
> 2-11-16 Yayoi, Bunkyo-ku, Tokyo 113-8658, Japan
> tel: +81-3-5841-0540
>   * in National Institute of Informatics
> 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo 101-8430, Japan
> tel: +81-3-4212-2412
> http://graal.ens-lyon.fr/~ycaniou/
>
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Problem with mpi_comm_spawn_multiple

2010-05-25 Thread Jeff Squyres
(somehow I apparently never sent this mail!  It's been sitting in my drafts 
folder for weeks.  To followup what I said below: I sent a question about this 
issue to the MPI-3 Fortran working group -- to see exactly what *should* be the 
right thing to do: http://lists.mpi-forum.org/mpi3-fortran/2010/05/0830.php)


On May 7, 2010, at 6:30 PM, Noam Bernstein wrote:

> As with every array in Fortran, arrays of strings
> are contiguous in memory, and presumably the end of string (1,1)
> is right before the beginning of string(2,1), etc.

Yep -- got all that -- we have some f2c string translation routines in OMPI for 
just this purpose.

But MPI_COMM_SPAWN_MULTIPLE is a little unique in that it takes a 2D array of 
character strings.  We know the first dimension; it has to be the same as the 
"count" argument to MPI_COMM_SPAWN_MULTIPLE.  And by the hidden argument, we 
know the length of all of the strings.  But we don't know the 2nd dimension of 
the array.

So OMPI searches for it by looking for an empty string (i.e., all spaces).  My 
gfortran 4.1 isn't doing that, although my ifort 11.1 and pgf90 10.0 are.  In 
my small example that I sent, it *is* apparently doing that for Andrew's ifort 
9.0, but it (apparently) isn't in his production tests.  Sigh.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/




Re: [OMPI users] Problem with mpi_comm_spawn_multiple

2010-05-25 Thread Jeff Squyres
Ah ha!

It turns out that the MPI spec specifically states that the user's code has to 
add a blank string at the end of the array to indicate the end.  Specifically 
(as was just quoted to me), MPI-2.2, p310:18-25 defines:

"The argv argument argv is an array of strings containing arguments that are 
passed to
the program. The first element of argv is the first argument passed to command, 
not, as
is conventional in some contexts, the command itself. The argument list is 
terminated by
NULL in C and C++ and an empty string in Fortran. In Fortran, leading and 
trailing spaces
are always stripped, so that a string consisting of all spaces is considered an 
empty string.
The constant MPI_ARGV_NULL may be used in C, C++ and Fortran to indicate an 
empty
argument list. In C and C++, this constant is the same as NULL."

This definition is also valid for MPI_COMM_SPAWN_MULTIPLE.

OMPI does mistakenly assume that all your argv have the same number of tokens; 
I'll fix that.



On May 25, 2010, at 11:55 AM, Jeff Squyres (jsquyres) wrote:

> (somehow I apparently never sent this mail!  It's been sitting in my drafts 
> folder for weeks.  To followup what I said below: I sent a question about 
> this issue to the MPI-3 Fortran working group -- to see exactly what *should* 
> be the right thing to do: 
> http://lists.mpi-forum.org/mpi3-fortran/2010/05/0830.php)
> 
> 
> On May 7, 2010, at 6:30 PM, Noam Bernstein wrote:
> 
> > As with every array in Fortran, arrays of strings
> > are contiguous in memory, and presumably the end of string (1,1)
> > is right before the beginning of string(2,1), etc.
> 
> Yep -- got all that -- we have some f2c string translation routines in OMPI 
> for just this purpose.
> 
> But MPI_COMM_SPAWN_MULTIPLE is a little unique in that it takes a 2D array of 
> character strings.  We know the first dimension; it has to be the same as the 
> "count" argument to MPI_COMM_SPAWN_MULTIPLE.  And by the hidden argument, we 
> know the length of all of the strings.  But we don't know the 2nd dimension 
> of the array.
> 
> So OMPI searches for it by looking for an empty string (i.e., all spaces).  
> My gfortran 4.1 isn't doing that, although my ifort 11.1 and pgf90 10.0 are.  
> In my small example that I sent, it *is* apparently doing that for Andrew's 
> ifort 9.0, but it (apparently) isn't in his production tests.  Sigh.
> 
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
> 
> 
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users
> 


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/