[OMPI users] How to reduce Isend & Irecv bandwidth?

2013-05-01 Thread Thomas Watson
Hi, I have a program where each MPI rank hosts a set of data blocks. After doing computation over *some of* its local data blocks, each MPI rank needs to exchange data with other ranks. Note that the computation may involve only a subset of the data blocks on a MPI rank. The data exchange is achie

Re: [OMPI users] How to reduce Isend & Irecv bandwidth?

2013-05-01 Thread Gus Correa
Maybe start the data exchange by sending a (presumably short) list/array/index-function of the dirty/not-dirty blocks status (say, 0=not-dirty,1=dirty), then putting if conditionals before the Isend/Irecv so that only dirty blocks are exchanged? I hope this helps, Gus Correa On 05/01/2013 01:2

Re: [OMPI users] How to reduce Isend & Irecv bandwidth?

2013-05-01 Thread Thomas Watson
Hi Gus, Thanks for your suggestion! The problem of this two-phased data exchange is as follows. Each rank can have data blocks that will be exchanged to potentially all other ranks. So if a rank needs to tell all the other ranks about which blocks to receive, it would require an all-to-all collec

Re: [OMPI users] How to reduce Isend & Irecv bandwidth?

2013-05-01 Thread Gus Correa
Hi Thomas/Jacky Maybe using MPI_Probe (and maybe also MPI_Cancel) to probe the message size, and receive only those with size>0? Anyway, I'm just code-guessing. I hope it helps, Gus Correa On 05/01/2013 05:14 PM, Thomas Watson wrote: Hi Gus, Thanks for your suggestion! The problem of this tw

Re: [OMPI users] How to reduce Isend & Irecv bandwidth?

2013-05-01 Thread Aurélien Bouteiller
Hi Jacky, 1. If you do not post a matching send, the wait(all) on the recv will stall forever. 2. You can match a recv(count, tag, src) with a send(0, tag, dst). The recv will complete, the status can be inspected to verify how many bytes have actually been received. It is illegal to send mor

Re: [OMPI users] How to reduce Isend & Irecv bandwidth?

2013-05-01 Thread Thomas Watson
Hi Aurelien, Excellent! Point 2) is exactly what I need - no data is actually sent and Irecv completes normally. Thanks! Jacky On Wed, May 1, 2013 at 6:29 PM, Aurélien Bouteiller wrote: > Hi Jacky, > > 1. If you do not post a matching send, the wait(all) on the recv will > stall forever. > 2.