Re: [OMPI users] Partitioning problem set data

[Eugene Loh]

The reason so many different distributions are described is because what is optimal depends so much on your own case. Even if one disregards CYCLIC axes, there are still all those BLOCK choices you mention.  It isn't just a matter of choosing which axes will be * since * is just a special case of BLOCK.  You have to choose the shape or aspect ratio of the subgrids. In some problems, it is desirable to minimize the surface-to-volume ratio of each local subgrid so that interprocess communication costs are minimized relative to the computational work each process has to do.  So, you would want to make each subgrid as "cubic" as possible.  Some sort of BLOCK,BLOCK,BLOCK. The "physics" may argue otherwise.  Not all directions may be the same.  E.g., in atmospheric models, the vertical direction is very different from the horizontal ones. Algorithms may also drive your choice.  E.g., for multidimensional FFTs, you might want one axis to be local.  Then, you would transpose axes to make another one local. You might also want the "innermost axis" (in the process's linear address space) to be as long as possible to benefit from software/hardware vectorization of computationally expensive loops. Lots of choices.  It depends on your problem. Alexandru Blidaru wrote: If there is an already existing implementation of the *Block or Block* methods that splits the array and sends the individual pieces to the proper nodes, can you point me to it please?  On Tue, Jul 20, 2010 at 9:52 AM, Alexandru Blidaru <alexs...@gmail.com> wrote: I have a 3D array, which I need to split into equal n parts, so that each part would run on a different node. I found the picture in the attachment from this website (https://computing.llnl.gov/tutorials/parallel_comp/#DesignPartitioning) on the different ways to partition data. I am interested in the block methods, as the cyclic methods wouldn't really work for me at all. Obviously the *, BLOCK and the BLOCK, * methods would be really easy to implement for 3D arrays, assuming that the 2D picture would be looking at the array from the top. My question is if there are other better ways to do it from a performance standpoint?