[OMPI users] Type struct question

[Thomas Anderson]

I read doc explaining struct at
http://www.open-mpi.org/doc/v1.4/man3/MPI_Type_struct.3.php

I haven't used mpi so I have a newbie question.

In the example function MPI_Type_struct(3, {2, 1, 3}, {0, 16, 26},
{MPI_FLOAT, type1, MPI_CHAR} ) is executed where type1 is consisted of
{(double,0)(char,8)}. The user defined datatype has extent 16 with
displacement at 16. However, the third block starts from displacement
26. This seems to me the end of type1 (at 32) would overlap the
displacement of MPI_CHAR (at 26). How does MPI solve this issue? Do
they simply ignore the rest space because it is only char type in
type1? Or if the type1 is defined by {(double, 0) (double, 8)}, won't
the data in the second block be truncated?

Thanks.

Re: [OMPI users] Type struct question

[George Bosilca]

On Nov 11, 2011, at 06:20 , Thomas Anderson wrote:

> I read doc explaining struct at
> http://www.open-mpi.org/doc/v1.4/man3/MPI_Type_struct.3.php
> 
> I haven't used mpi so I have a newbie question.
> 
> In the example function MPI_Type_struct(3, {2, 1, 3}, {0, 16, 26},
> {MPI_FLOAT, type1, MPI_CHAR} ) is executed where type1 is consisted of
> {(double,0)(char,8)}. The user defined datatype has extent 16 with
> displacement at 16. However, the third block starts from displacement
> 26. This seems to me the end of type1 (at 32) would overlap the
> displacement of MPI_CHAR (at 26). How does MPI solve this issue? Do
> they simply ignore the rest space because it is only char type in
> type1? Or if the type1 is defined by {(double, 0) (double, 8)}, won't
> the data in the second block be truncated?

If the overlap is on the gaps around the datatype, it is not that a big deal. I 
do not know how the other MPI libraries deal with this situation, but in Open 
MPI as long as there is no real overlap (in terms of real data) there will be 
no complaints from the library.

Your type1 contains the char at displacement 8. As, in the upper datatype, you 
start the type1 at position 16, the last real data will be the char at 
displacement 24, which is earlier than the double at displacement 26. So there 
is no overlap in this datatype.

If I dump with Open MPI this datatype, I get the following description:

Datatype 0x867270 id 69 size 20 align 8 opal_id 0 length 9 used 4
true_lb 0 true_ub 29 (true_extent 29) lb 0 ub 32 (extent 32)
nbElems 7 loops 0 flags 100 ()---GD--[---][---]
   contain OPAL_INT1 OPAL_FLOAT4 OPAL_FLOAT8 
--C---P-D--[---][---]OPAL_FLOAT4 count 2 disp 0x0 (0) extent 4 (size 8)
--C---P-D--[---][---]OPAL_FLOAT8 count 1 disp 0x10 (16) extent 8 (size 8)
--C---P-D--[---][---]  OPAL_INT1 count 1 disp 0x18 (24) extent 1 (size 1)
--C---P-D--[---][---]  OPAL_INT1 count 3 disp 0x1a (26) extent 1 (size 3)

which corroborate what I describe above.

  george.

> 
> Thanks.
> ___
> users mailing list
> us...@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

Re: [OMPI users] Process Migration

[Ioannis Papadopoulos]

Offtopic: You might want to have a look at AMPI: 
http://charm.cs.uiuc.edu/research/ampi


On 11/10/2011 10:30 AM, Mudassar Majeed wrote:
For example there are 10 nodes, and each node contains 20 cores. We 
will have 200 cores in total and let say there are 2000 MPI processes. 
We start the application with 10 MPI on each core. Let say Comm(Pi, 
Pj) denotes how much communication Pi and Pj make with each other and 
let say each process Pi has to communicate with few other processes 
Pj, Pk, Pl, Pm. Pz. Secondly let say Load(Pi) denotes the 
computational load of process Pi.


Now, we know that sending a message between two nodes is more 
expensive then sending a message within a node (two processes that 
communicate reside on the cores that exist in the same node). This is 
true atleast in my supercomputing centers that I use. In my previous 
work I only consider Load[ ] and not Comm[ ]. In that work, all the 
MPI processes calculate their new ranks and then call MPI_Comm_split 
with key = new_rank and color = 0. So all the processes get the new 
rank and then the actual data is provided to each process for 
computation. We have found that the total execution time decreases. 
Now we need to consider the communications as well. We will bring the 
computational load balance but those MPI which communicate more will 
be mapped to the same node (not necessarily same cores). I have solved 
this optimization problem using ILP and that shows good results. But 
the thing is, in the solution I have found that after applying ILP or 
my heuristic, the cores (on all nodes) will no longer contain same 
number of MPI processes (load and communications are balanced instead 
of count of MPI processes per core). So this means either I use 
process migration for few processes or I run more than 2000 (means at 
every core I run few more processes) so that at the end imbalance in 
the number or MPI processes per core can be achieved (to achieve 
balance in load and communications). I need your suggestions in these 
regards,


thanks and best regards,
Mudassar

*From:* Josh Hursey 
*To:* Open MPI Users 
*Cc:* Mudassar Majeed 
*Sent:* Thursday, November 10, 2011 5:11 PM
*Subject:* Re: [OMPI users] Process Migration

Note that the "migrate me from my current node to node " scenario
is covered by the migration API exported by the C/R infrastructure, as
I noted earlier.
  http://osl.iu.edu/research/ft/ompi-cr/api.php#api-cr_migrate

The "move rank N to node " scenario could probably be added as an
extension of this interface (since you can do that via the command
line now) if that is what you are looking for.

-- Josh

On Thu, Nov 10, 2011 at 11:03 AM, Ralph Castain > wrote:

> So what you are looking for is an MPI extension API that let's you say
> "migrate me from my current node to node "? Or do you have a 
rank that

> is the "master" that would order "move rank N to node "?
> Either could be provided, I imagine - just want to ensure I 
understand what

> you need. Can you pass along a brief description of the syntax and
> functionality you would need?
>
> On Nov 10, 2011, at 8:27 AM, Mudassar Majeed wrote:
>
> Thank you for your reply. In our previous publication, we have 
figured it

> out that run more than one processes on cores and balancing the
> computational load considerably reduces the total execution time. 
You know
> the MPI_Graph_create function, we created another function 
MPI_Load_create
> that maps the processes on cores such that balance of computational 
load can

> be achieved on cores. We were having some issues with increase in
> communication cost due to ranks rearrangements (due to 
MPI_Comm_split, with

> color=0), so in this research work we will see how can we balance both
> computation load on each core and communication load on each node. Those
> processes that communicate more will reside on the same node keeping the
> computational load balance over the cores. I solved this problem 
using ILP
> but ILP takes time and can't be used in run time so I am thinking 
about an
> heuristic. That's why I want to see if it is possible to migrate a 
process
> from one core to another or not. Then I will see how good my 
heuristic will

> be.
>
> thanks
> Mudassar
>
> 
> From: Jeff Squyres mailto:jsquy...@cisco.com>>
> To: Mudassar Majeed >; Open MPI Users

> mailto:us...@open-mpi.org>>
> Cc: Ralph Castain mailto:r...@open-mpi.org>>
> Sent: Thursday, November 10, 2011 2:19 PM
> Subject: Re: [OMPI users] Process Migration
>
> On Nov 10, 2011, at 8:11 AM, Mudassar Majeed wrote:
>
>> Thank you for your reply. I am implementing a load balancing 
function for
>> MPI, that will balance the computation load and the communication 
both at a

>> time. So my algorithm assumes that all the cores may at the end get
>> different number of processes to run.
>
> 

[OMPI users] mpi_sendrecv call example : case for a deadlock ?

[shankha]

Hi,
I have a example code using sendrecv. Could you please tell me if there is
a chance for a deadlock ?

   right =(myrank + 1) % size;
left  = myrank - 1;
if (left < 0)
left = size - 1;

MPI_Sendrecv(&s, 1, MPI_CHAR, left, 1231, &r, 1, MPI_CHAR, right,
1231, MPI_COMM_WORLD, &status_1[0]);

Node A Node B   Node C
  send to A
  recv from C

Each node is sending data to the node denoted by left and receiving data
from the node denoted by right. IS this is a case for deadlock.
Thanks for your help.

-- 
Thanks
Shankha Banerjee