I am replying to this via the daily-digest message I got. Sorry it wasn't
sooner... I didn't realize I was getting replies until I got the digest. Does
anyone know how to change it so I get the emails as you all send them?
>>Unless your computation is so "embarrassingly parallel" that each process
>>can operate from start to end totally independent from the others, you should
>>expect such an overlap, but you didn't tell what computation you want to do.
I am doing a Red-black Gauss-Seidel algorithm. The ghost points are what I was
trying to figure for moving this into the 3rd dimension. Thanks for adding some
concrete-ness to my idea of exactly how much overhead is involved. The test
domains I am computing on are on the order of 100*50*50 or so. This is why I am
trying to limit the overhead of the MPI communication. I am in the process of
finding out exactly how big the domains may become, so that I can adjust the
algorithm accordingly.
>> PS - The same book "MPI the complete reference" has a thorough description
>> of MPI types in Chapter 3.
You may want to create and use a MPI_TYPE_VECTOR with the appropriate count,
blocklength, and stride, to exchange all the "0..Z" overlap slices in a single
swoop.
(If I understood right, this is your main concern.)
This is exactly what I was trying to do before, but I soon discovered you have
to have your arrays tightly packed for this to work. Using a static array it is
fine, but for dynamically sized arrays, there has to be a separate malloc() in
C for the space, and then an array of pointers to that defined space to be able
to use three-bracket notation. Tricky that it works like that (at least it was
to me).
>>Also, I wonder why you want to decompose on both X and Y ("pencils"), and not
>>only X ("books"), which may give you a smaller/simpler domain decomposition
>>and communication footprint.
Whether you can or cannot do this way depends on your computation, which I
don't know about.
If I understand what you mean by pencils versus books, I don't think that will
work for these types of calculations will it? Maybe a better explanation of
what you mean by a pencil versus a book. If I was going to solve a sub-matrix
of the XY plane for all Z-values, what is that considered? My plan was to share
the appropriate top, bottom, left, and right neighbors (w.r.t. to the XY plane)
but NOT the above/below neighbors (since I am calculating from 0..Z completely,
in order).
