[OMPI users] Why communication performance change with binding PEs?

[Saliya Ekanayake]

Hi,

I am trying to understand this peculiar behavior where the communication
time in OpenMPI changes depending on the number of process elements (cores)
the process is bound to.

Is this expected?

Thank you,
saliya

-- 
Saliya Ekanayake
Ph.D. Candidate | Research Assistant
School of Informatics and Computing | Digital Science Center
Indiana University, Bloomington

Re: [OMPI users] Why communication performance change with binding PEs?

[Gilles Gouaillardet]

Can you please provide more details on your config, how test are 
performed and the results ?



to be fair, you should only compare cases in which mpi tasks are bound 
to the same sockets.


for example, if socket0 has core[0-7] and socket1 has core[8-15]

it is fair to compare {task0,task1} bound on

{0,8}, {[0-1],[8-9]}, {[0-7],[8-15]}

but it is unfair to compare

{0,1} and {0,8} or {[0-7],[8-15]}

since {0,1} does not involve traffic on the QPI, but {0,8} does.

depending on the btl you are using, it might involve or not an other 
"helper" thread.
if your task is bound on one core, and assuming there is no SMT, then 
the task and the helper do time sharing.
but if the task is bound on more than one core, then the task and the 
helper run in parallel.



Cheers,

Gilles
On 6/23/2016 1:21 PM, Saliya Ekanayake wrote:

Hi,

I am trying to understand this peculiar behavior where the 
communication time in OpenMPI changes depending on the number of 
process elements (cores) the process is bound to.


Is this expected?

Thank you,
saliya

--
Saliya Ekanayake
Ph.D. Candidate | Research Assistant
School of Informatics and Computing | Digital Science Center
Indiana University, Bloomington



___
users mailing list
us...@open-mpi.org
Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2016/06/29523.php

Re: [OMPI users] Shared Libraries

[Ralph Castain]

Wow, I haven’t encountered Forth in over 20 years! Though I confess I used to 
program in it myself back in my control days.

IIRC, you would need to write a wrapper to let Forth access C-based functions, 
yes? You could configure and build OMPI as a 32-bit library, and libmpi.so is 
C, so that isn’t an issue.

What compiler are you planning to use (as the cmd line options may vary a bit)?


> On Jun 22, 2016, at 6:34 PM, Richard C. Wagner  wrote:
> 
> Hi Everyone:
> 
> I'm trying to employ MPI in an unconventional programming language, Forth, 
> running over Debian Linux.  The Forth I have can import a Linux shared 
> library in the .so file format and then compile in the executable functions 
> as externals.  The question: how to do it?  I'm looking to access MPI_Init, 
> MPI_Finalize, MPI_Comm_size, MPI_Comm_rank, MPI_Waitany, MPI_Isend, MPI_Irecv 
> plus other functions as needed.  The Forth is a 32-bit executable, so I need 
> to include functions from a 32-bit library.  Is there a single 32-bit MPI 
> library where I can find these functions?  Or would it make sense to create 
> my own, 32-bit library using C?
> 
> Any ideas?
> 
> 
> Regards,
> 
> Rich Wagner
> Aerospace Engineer
> Graduate Student
> Fort Collins, CO, USA 
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2016/06/29522.php

[OMPI users] Fw: OpenSHMEM Runtime Error

[RYAN RAY]

From: "RYAN RAY"Sent: Wed, 22 Jun 2016 
14:32:33To: "users"Subject: OpenSHMEM 
Runtime ErrorI have installed openmpi-1.10.1 in a system and while executing 
one of the example codes of OpenSHMEM I am getting an error. The snapshot of 
the error is attached. Can anybody please help me to sort out this 
error?RegardsRyan Saptarshi Ray 

Re: [OMPI users] Fw: OpenSHMEM Runtime Error

[Jeff Squyres (jsquyres)]

Ryan --

Did you try the suggestions listed in the help message?


> On Jun 23, 2016, at 1:24 AM, RYAN RAY  wrote:
> 
> 
> 
> From: "RYAN RAY"
> Sent: Wed, 22 Jun 2016 14:32:33
> To: "users"
> Subject: OpenSHMEM Runtime Error
> 
> I have installed openmpi-1.10.1 in a system and while executing one of the 
> example codes of OpenSHMEM I am getting an error. The snapshot of the error 
> is attached. Can anybody please help me to sort out this error?
> 
> Regards
> 
> Ryan Saptarshi Ray
> 
>  
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2016/06/29526.php


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI users] Shared Libraries

[Jeff Squyres (jsquyres)]

Greetings Richard.

Yes, that certainly is unusual.  :-)

Here's my advice:

- Configure Open MPI with the --disable-dlopen flag.  This will slurp in all of 
Open MPI's plugins into the main library, and make things considerably simpler 
for you.

- Build Open MPI in a 32 bit mode -- e.g., supply CFLAGS=-m32 on the configure 
command line (or whatever flags are relevant for your compiler).

Both of these taken together:

  ./configure CFLAGS=-m32 --disable-dlopen ...

- Then you should be able to dlopen libmpi.so and dlsym the symbols that you 
expect (e.g., MPI_Init and friends).


> On Jun 22, 2016, at 9:34 PM, Richard C. Wagner  wrote:
> 
> Hi Everyone:
> 
> I'm trying to employ MPI in an unconventional programming language, Forth, 
> running over Debian Linux.  The Forth I have can import a Linux shared 
> library in the .so file format and then compile in the executable functions 
> as externals.  The question: how to do it?  I'm looking to access MPI_Init, 
> MPI_Finalize, MPI_Comm_size, MPI_Comm_rank, MPI_Waitany, MPI_Isend, MPI_Irecv 
> plus other functions as needed.  The Forth is a 32-bit executable, so I need 
> to include functions from a 32-bit library.  Is there a single 32-bit MPI 
> library where I can find these functions?  Or would it make sense to create 
> my own, 32-bit library using C?
> 
> Any ideas?
> 
> 
> Regards,
> 
> Rich Wagner
> Aerospace Engineer
> Graduate Student
> Fort Collins, CO, USA 
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2016/06/29522.php


-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI users] Why communication performance change with binding PEs?

[Saliya Ekanayake]

Thank you, Gilles for the quick response. The code comes from a clustering
application, bu let me try to explain simply what the pattern is. It's a
bit long than I expected.



The program has the pattern BSP pattern with *compute()* followed by
collective *allreduce()* And it does many iterations over these two.

Each process is a Java process with just the main thread. However in Java
the process and main thread have their own PIDs and act as two LWPs in
Linux.

Now, let's take two binding scenarios. For simplicity, I'll assume a node
with 2 sockets each with 4-cores. The real one I ran has 2 sockets with 12
cores each.

1. *--map-by ppr:8:node:PE=1 --bind-to core* results in something like
below.

[image: Inline image 3]
where each process is bound to 1 core. The blue dots show the main thread
in Java. It too is bound to the same core as its parent process by default.

2. *--map-by ppr:8:node  --bind-to none * This is similar to 1, but now
processes are not bound (or bound to all cores). However, from the program,
we *explicitly bind its main thread to 1 core*. It gives something like
below.

[image: Inline image 4]
The results we got suggest approach 2 gives better communication
performance than 1. The btl used is openib. Here's a graph showing the
variation in timings. It shows for other cases that use more than 1 thread
to do the computation as well. In all patterns communication is done
through the main thread only.

What is peculiar is the two points within the dotted circle. Intuitively
they should overlap as it only has the main thread in each Java process and
that main is bound to 1 core. The difference is how the parent process is
bound with MPI. The red line is for *Case 1* above and the blue is for *Case
2*

The green line is when both parent process and threads are unbound.


[image: Inline image 6]







On Thu, Jun 23, 2016 at 12:36 AM, Gilles Gouaillardet 
wrote:

> Can you please provide more details on your config, how test are performed
> and the results ?
>
>
> to be fair, you should only compare cases in which mpi tasks are bound to
> the same sockets.
>
> for example, if socket0 has core[0-7] and socket1 has core[8-15]
>
> it is fair to compare {task0,task1} bound on
>
> {0,8}, {[0-1],[8-9]}, {[0-7],[8-15]}
>
> but it is unfair to compare
>
> {0,1} and {0,8} or {[0-7],[8-15]}
>
> since {0,1} does not involve traffic on the QPI, but {0,8} does.
> depending on the btl you are using, it might involve or not an other
> "helper" thread.
> if your task is bound on one core, and assuming there is no SMT, then the
> task and the helper do time sharing.
> but if the task is bound on more than one core, then the task and the
> helper run in parallel.
>
>
> Cheers,
>
> Gilles
>
> On 6/23/2016 1:21 PM, Saliya Ekanayake wrote:
>
> Hi,
>
> I am trying to understand this peculiar behavior where the communication
> time in OpenMPI changes depending on the number of process elements (cores)
> the process is bound to.
>
> Is this expected?
>
> Thank you,
> saliya
>
> --
> Saliya Ekanayake
> Ph.D. Candidate | Research Assistant
> School of Informatics and Computing | Digital Science Center
> Indiana University, Bloomington
>
>
>
> ___
> users mailing listus...@open-mpi.org
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2016/06/29523.php
>
>
>
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2016/06/29524.php
>



-- 
Saliya Ekanayake
Ph.D. Candidate | Research Assistant
School of Informatics and Computing | Digital Science Center
Indiana University, Bloomington

Re: [OMPI users] Why communication performance change with binding PEs?

[Gilles Gouaillardet]

Java uses *many* threads, simply

ls /proc//tasks

and you will be amazed at how many threads are used.
Here is my guess,


from the point of view of a given MPI process :

in case 1, the main thread and all the other threads do time sharing, so 
basically, when an other thread is working, the main thread is blocked.


in case 2, some parallelism is possible if an other MPI task is sleeping 
: main thread is running, and an other thread is running on an other core


in case 3, the main thread can move from on core to an other
=> cache flush
=> QPI access if used memory is no more local
so though there is more opportunity for parallelism, process migration 
can slow down everything



bottom line, event with one thread, case 1 and case 2 are quite 
different because Java uses so many threads per process, so i am not so 
surprised with the difference in performance.


if you have any chance, i suggest you write a similar program in C.
since only a few threads are use per process, i guess case 1 and case 2 
will become pretty close.


i also suggest that for cases 2 and 3, you bind processes to a socket 
instead of no binding at all


Cheers,

Gilles

On 6/23/2016 2:41 PM, Saliya Ekanayake wrote:
Thank you, Gilles for the quick response. The code comes from a 
clustering application, bu let me try to explain simply what the 
pattern is. It's a bit long than I expected.




The program has the pattern BSP pattern with /compute()/ followed by 
collective /allreduce()/ And it does many iterations over these two.


Each process is a Java process with just the main thread. However in 
Java the process and main thread have their own PIDs and act as two 
LWPs in Linux.


Now, let's take two binding scenarios. For simplicity, I'll assume a 
node with 2 sockets each with 4-cores. The real one I ran has 2 
sockets with 12 cores each.


1. *--map-by ppr:8:node:PE=1 --bind-to core* results in something like 
below.


Inline image 3
where each process is bound to 1 core. The blue dots show the main 
thread in Java. It too is bound to the same core as its parent process 
by default.


2. *--map-by ppr:8:node  --bind-to none * This is similar to 1, but 
now processes are not bound (or bound to all cores). However, from the 
program, we *explicitly bind its main thread to 1 core*. It gives 
something like below.


Inline image 4
The results we got suggest approach 2 gives better communication 
performance than 1. The btl used is openib. Here's a graph showing the 
variation in timings. It shows for other cases that use more than 1 
thread to do the computation as well. In all patterns communication is 
done through the main thread only.


What is peculiar is the two points within the dotted circle. 
Intuitively they should overlap as it only has the main thread in each 
Java process and that main is bound to 1 core. The difference is how 
the parent process is bound with MPI. The red line is for *Case 
1* above and the blue is for *Case 2*

*
*
The green line is when both parent process and threads are unbound.


Inline image 6


/
/
/
/



On Thu, Jun 23, 2016 at 12:36 AM, Gilles Gouaillardet 
mailto:gil...@rist.or.jp>> wrote:


Can you please provide more details on your config, how test are
performed and the results ?


to be fair, you should only compare cases in which mpi tasks are
bound to the same sockets.

for example, if socket0 has core[0-7] and socket1 has core[8-15]

it is fair to compare {task0,task1} bound on

{0,8}, {[0-1],[8-9]}, {[0-7],[8-15]}

but it is unfair to compare

{0,1} and {0,8} or {[0-7],[8-15]}

since {0,1} does not involve traffic on the QPI, but {0,8} does.

depending on the btl you are using, it might involve or not an
other "helper" thread.
if your task is bound on one core, and assuming there is no SMT,
then the task and the helper do time sharing.
but if the task is bound on more than one core, then the task and
the helper run in parallel.


Cheers,

Gilles

On 6/23/2016 1:21 PM, Saliya Ekanayake wrote:

Hi,

I am trying to understand this peculiar behavior where the
communication time in OpenMPI changes depending on the number of
process elements (cores) the process is bound to.

Is this expected?

Thank you,
saliya

-- 
Saliya Ekanayake

Ph.D. Candidate | Research Assistant
School of Informatics and Computing | Digital Science Center
Indiana University, Bloomington



___
users mailing list
us...@open-mpi.org 
Subscription:https://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this 
post:http://www.open-mpi.org/community/lists/users/2016/06/29523.php



___
users mailing list
us...@open-mpi.org 
Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post:
   

Re: [OMPI users] Why communication performance change with binding PEs?

[Saliya Ekanayake]

Thank you, this is really helpful. Yes, the other bookkeeping threads of
Java were what I worried too.

I think I can extract a part to make a  c program to check.

I've got a quick question. Besides theses time sharing constraints, does
number of cores has any significance to MPI's communication decisions?
On Jun 23, 2016 2:18 AM, "Gilles Gouaillardet"  wrote:

> Java uses *many* threads, simply
>
> ls /proc//tasks
> and you will be amazed at how many threads are used.
> Here is my guess,
>
>
> from the point of view of a given MPI process :
>
> in case 1, the main thread and all the other threads do time sharing, so
> basically, when an other thread is working, the main thread is blocked.
>
> in case 2, some parallelism is possible if an other MPI task is sleeping :
> main thread is running, and an other thread is running on an other core
>
> in case 3, the main thread can move from on core to an other
> => cache flush
> => QPI access if used memory is no more local
> so though there is more opportunity for parallelism, process migration can
> slow down everything
>
>
> bottom line, event with one thread, case 1 and case 2 are quite different
> because Java uses so many threads per process, so i am not so surprised
> with the difference in performance.
>
> if you have any chance, i suggest you write a similar program in C.
> since only a few threads are use per process, i guess case 1 and case 2
> will become pretty close.
>
> i also suggest that for cases 2 and 3, you bind processes to a socket
> instead of no binding at all
>
> Cheers,
>
> Gilles
>
> On 6/23/2016 2:41 PM, Saliya Ekanayake wrote:
>
> Thank you, Gilles for the quick response. The code comes from a clustering
> application, bu let me try to explain simply what the pattern is. It's a
> bit long than I expected.
>
>
>
> The program has the pattern BSP pattern with *compute()* followed by
> collective *allreduce()* And it does many iterations over these two.
>
> Each process is a Java process with just the main thread. However in Java
> the process and main thread have their own PIDs and act as two LWPs in
> Linux.
>
> Now, let's take two binding scenarios. For simplicity, I'll assume a node
> with 2 sockets each with 4-cores. The real one I ran has 2 sockets with 12
> cores each.
>
> 1. *--map-by ppr:8:node:PE=1 --bind-to core* results in something like
> below.
>
> [image: Inline image 3]
> where each process is bound to 1 core. The blue dots show the main thread
> in Java. It too is bound to the same core as its parent process by default.
>
> 2. *--map-by ppr:8:node  --bind-to none * This is similar to 1, but now
> processes are not bound (or bound to all cores). However, from the program,
> we *explicitly bind its main thread to 1 core*. It gives something like
> below.
>
> [image: Inline image 4]
> The results we got suggest approach 2 gives better communication
> performance than 1. The btl used is openib. Here's a graph showing the
> variation in timings. It shows for other cases that use more than 1 thread
> to do the computation as well. In all patterns communication is done
> through the main thread only.
>
> What is peculiar is the two points within the dotted circle. Intuitively
> they should overlap as it only has the main thread in each Java process and
> that main is bound to 1 core. The difference is how the parent process is
> bound with MPI. The red line is for *Case 1* above and the blue is for *Case
> 2*
>
> The green line is when both parent process and threads are unbound.
>
>
> [image: Inline image 6]
>
>
>
>
>
>
>
> On Thu, Jun 23, 2016 at 12:36 AM, Gilles Gouaillardet 
> wrote:
>
>> Can you please provide more details on your config, how test are
>> performed and the results ?
>>
>>
>> to be fair, you should only compare cases in which mpi tasks are bound to
>> the same sockets.
>>
>> for example, if socket0 has core[0-7] and socket1 has core[8-15]
>>
>> it is fair to compare {task0,task1} bound on
>>
>> {0,8}, {[0-1],[8-9]}, {[0-7],[8-15]}
>>
>> but it is unfair to compare
>>
>> {0,1} and {0,8} or {[0-7],[8-15]}
>>
>> since {0,1} does not involve traffic on the QPI, but {0,8} does.
>> depending on the btl you are using, it might involve or not an other
>> "helper" thread.
>> if your task is bound on one core, and assuming there is no SMT, then the
>> task and the helper do time sharing.
>> but if the task is bound on more than one core, then the task and the
>> helper run in parallel.
>>
>>
>> Cheers,
>>
>> Gilles
>>
>> On 6/23/2016 1:21 PM, Saliya Ekanayake wrote:
>>
>> Hi,
>>
>> I am trying to understand this peculiar behavior where the communication
>> time in OpenMPI changes depending on the number of process elements (cores)
>> the process is bound to.
>>
>> Is this expected?
>>
>> Thank you,
>> saliya
>>
>> --
>> Saliya Ekanayake
>> Ph.D. Candidate | Research Assistant
>> School of Informatics and Computing | Digital Science Center
>> Indiana University, Bloomington
>>
>>
>>
>> 

Re: [OMPI users] Why communication performance change with binding PEs?

[Jeff Squyres (jsquyres)]

On Jun 23, 2016, at 8:20 AM, Saliya Ekanayake  wrote:
> 
> I've got a quick question. Besides theses time sharing constraints, does 
> number of cores has any significance to MPI's communication decisions?

Open MPI doesn't use the number of cores available to it in any calculations / 
algorithm selection in the MPI layer.

The locality of the running thread may have an impact on performance (and/or 
algorithm selection), though.  E.g., if a thread is accidentally accessing 
NUMA-remote memory.

-- 
Jeff Squyres
jsquy...@cisco.com
For corporate legal information go to: 
http://www.cisco.com/web/about/doing_business/legal/cri/

Re: [OMPI users] Fw: OpenSHMEM Runtime Error

[Joshua Ladd]

Ryan,

Four suggestions are provided in the help output. Please try these.

Josh

On Thu, Jun 23, 2016 at 1:25 AM, Jeff Squyres (jsquyres)  wrote:

> Ryan --
>
> Did you try the suggestions listed in the help message?
>
>
> > On Jun 23, 2016, at 1:24 AM, RYAN RAY  wrote:
> >
> >
> >
> > From: "RYAN RAY"
> > Sent: Wed, 22 Jun 2016 14:32:33
> > To: "users"
> > Subject: OpenSHMEM Runtime Error
> >
> > I have installed openmpi-1.10.1 in a system and while executing one of
> the example codes of OpenSHMEM I am getting an error. The snapshot of the
> error is attached. Can anybody please help me to sort out this error?
> >
> > Regards
> >
> > Ryan Saptarshi Ray
> >
> >
> > ___
> > users mailing list
> > us...@open-mpi.org
> > Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
> > Link to this post:
> http://www.open-mpi.org/community/lists/users/2016/06/29526.php
>
>
> --
> Jeff Squyres
> jsquy...@cisco.com
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post:
> http://www.open-mpi.org/community/lists/users/2016/06/29527.php
>

[OMPI users] [[INVALID], INVALID] ORTE_ERROR_LOG: Not found in file ess_env_module.c at line 367

[Blair Climenhaga]

Hello Everyone!

I recently downloaded OpenFOAM and while attempting to use its parallel 
features (which use mpi) I recieve the following error:

su2@su2-HP:~/OpenFOAM/su2-3.0.1/run/tutorials/incompressible/simpleFoam/motorBike_baseCase$
 mpirun -np 4 simpleFoam -parallel


[su2-HP:21015] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
ess_env_module.c at line 367


[su2-HP:21016] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
ess_env_module.c at line 367


[su2-HP:21013] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
ess_env_module.c at line 367


[su2-HP:21014] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
ess_env_module.c at line 367


[su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720


[su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720


[su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720


[su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720

I have double-checked with colleagues that all OpenFOAM functions were correct 
so the only two remaining causes are that the mpi was not installed correctly 
and or that it is not correctly linked to the OpenFOAM. From the errors shown 
above, does anyone have an idea of which it may be? Or if it may be another 
problem entirely. Thank you everyone for your time and for any help you may be 
able to give. 

All the best,
Blair Climenhaga
  

Re: [OMPI users] [[INVALID], INVALID] ORTE_ERROR_LOG: Not found in file ess_env_module.c at line 367

[Ralph Castain]

Looks like you are getting mixes of OMPI installations between the nodes - try 
ensuring that the PATH and LD_LIBRARY_PATH are correct on all the nodes

> On Jun 23, 2016, at 11:48 AM, Blair Climenhaga  
> wrote:
> 
> Hello Everyone!
> 
> I recently downloaded OpenFOAM and while attempting to use its parallel 
> features (which use mpi) I recieve the following error:
> 
> su2@su2-HP:~/OpenFOAM/su2-3.0.1/run/tutorials/incompressible/simpleFoam/motorBike_baseCase$
>  mpirun -np 4 simpleFoam -parallel
> [su2-HP:21015] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
> ess_env_module.c at line 367
> [su2-HP:21016] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
> ess_env_module.c at line 367
> [su2-HP:21013] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
> ess_env_module.c at line 367
> [su2-HP:21014] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
> ess_env_module.c at line 367
> [su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
> [su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
> [su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
> [su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
> 
> I have double-checked with colleagues that all OpenFOAM functions were 
> correct so the only two remaining causes are that the mpi was not installed 
> correctly and or that it is not correctly linked to the OpenFOAM. From the 
> errors shown above, does anyone have an idea of which it may be? Or if it may 
> be another problem entirely. Thank you everyone for your time and for any 
> help you may be able to give. 
> 
> All the best,
> Blair Climenhaga
> ___
> users mailing list
> us...@open-mpi.org 
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users 
> 
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2016/06/29534.php 
> 

Re: [OMPI users] [[INVALID], INVALID] ORTE_ERROR_LOG: Not found in file ess_env_module.c at line 367

[Blair Climenhaga]

Hi Ralph,

Thank you for your reply. How would I check that the PATH and LD_LIBRARY_PATH 
are correct on all nodes? I have a feeling that this is a likely problem though 
as the computer I am using has had many iterations of MPI installed on it and 
likely in different locations. 

All the best,
Blair Climenhaga

From: r...@open-mpi.org
List-Post: users@lists.open-mpi.org
Date: Thu, 23 Jun 2016 11:59:56 -0700
To: us...@open-mpi.org
Subject: Re: [OMPI users] [[INVALID],   INVALID] ORTE_ERROR_LOG: Not found in 
file ess_env_module.c at line 367

Looks like you are getting mixes of OMPI installations between the nodes - try 
ensuring that the PATH and LD_LIBRARY_PATH are correct on all the nodes
On Jun 23, 2016, at 11:48 AM, Blair Climenhaga  
wrote:Hello Everyone!

I recently downloaded OpenFOAM and while attempting to use its parallel 
features (which use mpi) I recieve the following error:

su2@su2-HP:~/OpenFOAM/su2-3.0.1/run/tutorials/incompressible/simpleFoam/motorBike_baseCase$
 mpirun -np 4 simpleFoam -parallel
[su2-HP:21015] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
ess_env_module.c at line 367
[su2-HP:21016] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
ess_env_module.c at line 367
[su2-HP:21013] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
ess_env_module.c at line 367
[su2-HP:21014] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
ess_env_module.c at line 367
[su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
[su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
[su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
[su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720

I have double-checked with colleagues that all OpenFOAM functions were correct 
so the only two remaining causes are that the mpi was not installed correctly 
and or that it is not correctly linked to the OpenFOAM. From the errors shown 
above, does anyone have an idea of which it may be? Or if it may be another 
problem entirely. Thank you everyone for your time and for any help you may be 
able to give. 

All the best,
Blair Climenhaga
___users mailing 
listusers@open-mpi.orgSubscription: 
https://www.open-mpi.org/mailman/listinfo.cgi/usersLink to this post: 
http://www.open-mpi.org/community/lists/users/2016/06/29534.php

___
users mailing list
us...@open-mpi.org
Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
Link to this post: 
http://www.open-mpi.org/community/lists/users/2016/06/29535.php 
   

Re: [OMPI users] [[INVALID], INVALID] ORTE_ERROR_LOG: Not found in file ess_env_module.c at line 367

[Ralph Castain]

One easy solution: configure OMPI with —enable-orterun-prefix-by-default and it 
will ensure that all the launched daemons and procs have the right setting on 
the backend nodes. Or you can ssh to each node and print the relevant envars 
and see what they say.


> On Jun 23, 2016, at 12:19 PM, Blair Climenhaga  
> wrote:
> 
> Hi Ralph,
> 
> Thank you for your reply. How would I check that the PATH and LD_LIBRARY_PATH 
> are correct on all nodes? I have a feeling that this is a likely problem 
> though as the computer I am using has had many iterations of MPI installed on 
> it and likely in different locations. 
> 
> All the best,
> Blair Climenhaga
> 
> From: r...@open-mpi.org
> Date: Thu, 23 Jun 2016 11:59:56 -0700
> To: us...@open-mpi.org
> Subject: Re: [OMPI users] [[INVALID], INVALID] ORTE_ERROR_LOG: Not found in 
> file ess_env_module.c at line 367
> 
> Looks like you are getting mixes of OMPI installations between the nodes - 
> try ensuring that the PATH and LD_LIBRARY_PATH are correct on all the nodes
> 
> On Jun 23, 2016, at 11:48 AM, Blair Climenhaga  > wrote:
> 
> Hello Everyone!
> 
> I recently downloaded OpenFOAM and while attempting to use its parallel 
> features (which use mpi) I recieve the following error:
> 
> su2@su2-HP:~/OpenFOAM/su2-3.0.1/run/tutorials/incompressible/simpleFoam/motorBike_baseCase$
>  mpirun -np 4 simpleFoam -parallel
> [su2-HP:21015] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
> ess_env_module.c at line 367
> [su2-HP:21016] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
> ess_env_module.c at line 367
> [su2-HP:21013] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
> ess_env_module.c at line 367
> [su2-HP:21014] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file 
> ess_env_module.c at line 367
> [su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
> [su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
> [su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
> [su2-HP:21011] tcp_peer_recv_connect_ack: invalid header type: -1721630720
> 
> I have double-checked with colleagues that all OpenFOAM functions were 
> correct so the only two remaining causes are that the mpi was not installed 
> correctly and or that it is not correctly linked to the OpenFOAM. From the 
> errors shown above, does anyone have an idea of which it may be? Or if it may 
> be another problem entirely. Thank you everyone for your time and for any 
> help you may be able to give. 
> 
> All the best,
> Blair Climenhaga
> ___
> users mailing list
> us...@open-mpi.org 
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users 
> 
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2016/06/29534.php 
> 
> 
> ___ users mailing list 
> us...@open-mpi.org  Subscription: 
> https://www.open-mpi.org/mailman/listinfo.cgi/users 
>  Link to this post: 
> http://www.open-mpi.org/community/lists/users/2016/06/29535.php 
> ___
> users mailing list
> us...@open-mpi.org 
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users 
> 
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2016/06/29536.php 
> 

[OMPI users] Fwd: Run MPI in Firewall Enviroment

[Wilkinson, Collin]

I realize that OpenMPI wasn't made to run in a Firewall environment. I'd
like to try to get it to run in said environment though. So what *exact*
ports do I need to open to be able to run in a firewall environment? And
how can I set MPI to run on said ports?

Any help I would really appreciate.

Thank You.

Re: [OMPI users] Fwd: Run MPI in Firewall Enviroment

[Ralph Castain]

Both the runtime and TCP BTL components accept port range definitions. All you 
have to do is tell us what those are, and then set your firewall to leave those 
ports open.

So the cmd line would look like: mpirun -mca oob_tcp_dynamic_ipv4_ports 
12345-12350 -mca btl_tcp_port_min_v4 34561 -mca btl_tcp_port_range_v4 100 ...

The two components are a little different. The oob_tcp one takes a range, while 
the btl_tcp one wants a starting port number and then the number of ports in 
the range. Still, the result is the same.

HTH
Ralph

> On Jun 23, 2016, at 7:26 PM, Wilkinson, Collin  wrote:
> 
> 
> I realize that OpenMPI wasn't made to run in a Firewall environment. I'd like 
> to try to get it to run in said environment though. So what exact ports do I 
> need to open to be able to run in a firewall environment? And how can I set 
> MPI to run on said ports?
> 
> Any help I would really appreciate.
> 
> Thank You.
> 
> ___
> users mailing list
> us...@open-mpi.org
> Subscription: https://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: 
> http://www.open-mpi.org/community/lists/users/2016/06/29538.php