On Fri, May 20, 2011 at 08:14:07AM -0400, Jeff Squyres wrote: > On May 20, 2011, at 6:23 AM, Jeff Squyres wrote: > > > Shouldn't ijlena and ijdisp be 1D arrays, not 2D arrays? > > Ok, if I convert ijlena and ijdisp to 1D arrays, I don't get the compile > error (even though they're allocatable -- so allocate was a red herring, > sorry). That's all that "use mpi" is complaining about -- that the function > signatures didn't match. > > use mpi is your friend -- even if you don't use F90 constructs much. > Compile-time checking is Very Good Thing (you were effectively "getting > lucky" by passing in the 2D arrays, I think). > > Attached is my final version. And with this version, I see the hang when > running it with the "T" parameter. > > That being said, I'm not an expert on the MPI IO stuff -- your code *looks* > right to me, but I could be missing something subtle in the interpretation of > MPI_FILE_SET_VIEW. I tried running your code with MPICH 1.3.2p1 and it also > hung. > > Rob (ROMIO guy) -- can you comment this code? Is it correct?
There's a kind of obscure but important rule in MPI-IO: the file view must describe monotonically non-decreasing offsets. the T type creates a file type with the following flattened representation (you can kind of think of the flattened representation as a type map, except everything is in terms of bytes): (0, 32), (96, 32), (32, 64) So, 32 bytes at offset 0, 32 bytes at offset 96 and 64 bytes at offset 32. That sort of looks like this: |xxxx~~~~~~~~~~~~zzzz~~~~yyyy| But you need the zzzz and yyyy pieces to be swapped in file view. It's an annoying part of the standard but as you can see if you violate that ROMIO will go off and spin in an infinite loop looking for the next piece of I/O (which in this case was "behind" the current piece). You can work around this by adjusting your memory datatype: data must be read off of the disk in this monotonically non-decreasing order but it can be jammed into memory any which way you want. ROMIO should be better about reporting file views that violate this part of the standard. We report it in a few places but clearly not enough. ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA
