Re: [OMPI users] 3D domain decomposition with MPI

[Eugene Loh]

It already seems that you have a good idea of what challenges you're facing.  So, I'm unclear which part you're asking about. Which cells do you need to update [x][y][z]?  It sounds like you need nearest neighbors.  So, one technique is to allocate on each process not just the subsection of data that will be computed by that process but also ghost cells. So, the first step would be to fill in ghost cells from neighboring processes.  You can either pack faces into contiguous arrays for sending (and unpacking on the other end), or you can use derived datatypes (the MPI standard is perhaps sufficiently readable that you could just go there). Once the ghost cells are packed, each process can compute its elements locally. Sorry if I missed the point of what you were asking. Cole, Derek E wrote: Hi all. I am relatively new to MPI, and so this may be covered somewhere else, but I can’t seem to find any links to tutorials mentioning any specifics, so perhaps someone here can help.   In C, I have a 3D array that I have dynamically allocated and access like Array[x][y][z]. I was hoping to calculate a subsection for each processor to work on,  of size nx in the x dimension, ny in the y dimension, and the full Z dimension. Starting at Array[sx][sy][0] and going to Array[ex][ey][z] where ey-sy=ny.   What is the best way to do this? I am able to calculate the neighboring processors and assign a sub-section of the XY dimensions to each processor, however I am having problems with sharing the border information of the arrays with the other processors. I don’t really want to have to do a MPI_Send for each of the 0..Z slices’s border information. I’d kind of like to process all of the Z, then share the full “face” of the border information with the neighbor processor. For example, if process 1 was the right neighbor of process zero, I’d want process zero to send Subarray[0..nx][ny][0..Z](the right most face) to processor 1’s left-most face..assuming the X-Y plane was your screen, and the Z dimension extended into the screen.   If anyone has any information that talks about how to use the MPI data types, or some other method, or wants to talk about how this might be done, I’m all ears.   I know it is hard to talk about without pictures, so if you all like, I can post a picture explaning what I want to do. Thanks!