Rob, Thanks for the clarification. I had seen that point about non-decreasing offsets in the standard and it was just beginning to dawn on me that maybe it was my problem. I will rethink my mapping strategy to comply with the restriction. Thanks again.
T. Rosmond On Tue, 2011-05-24 at 10:09 -0500, Rob Latham wrote: > On Fri, May 20, 2011 at 08:14:07AM -0400, Jeff Squyres wrote: > > On May 20, 2011, at 6:23 AM, Jeff Squyres wrote: > > > > > Shouldn't ijlena and ijdisp be 1D arrays, not 2D arrays? > > > > Ok, if I convert ijlena and ijdisp to 1D arrays, I don't get the compile > > error (even though they're allocatable -- so allocate was a red herring, > > sorry). That's all that "use mpi" is complaining about -- that the > > function signatures didn't match. > > > > use mpi is your friend -- even if you don't use F90 constructs much. > > Compile-time checking is Very Good Thing (you were effectively "getting > > lucky" by passing in the 2D arrays, I think). > > > > Attached is my final version. And with this version, I see the hang when > > running it with the "T" parameter. > > > > That being said, I'm not an expert on the MPI IO stuff -- your code *looks* > > right to me, but I could be missing something subtle in the interpretation > > of MPI_FILE_SET_VIEW. I tried running your code with MPICH 1.3.2p1 and it > > also hung. > > > > Rob (ROMIO guy) -- can you comment this code? Is it correct? > > There's a kind of obscure but important rule in MPI-IO: the file view > must describe monotonically non-decreasing offsets. > > the T type creates a file type with the following flattened > representation (you can kind of think of the flattened representation > as a type map, except everything is in terms of bytes): > > (0, 32), (96, 32), (32, 64) > > So, 32 bytes at offset 0, 32 bytes at offset 96 and 64 bytes at offset > 32. > > That sort of looks like this: > |xxxx~~~~~~~~~~~~zzzz~~~~yyyy| > > But you need the zzzz and yyyy pieces to be swapped in file view. > > It's an annoying part of the standard but as you can see if you > violate that ROMIO will go off and spin in an infinite loop looking > for the next piece of I/O (which in this case was "behind" the current > piece). > > You can work around this by adjusting your memory datatype: data must > be read off of the disk in this monotonically non-decreasing order but > it can be jammed into memory any which way you want. > > ROMIO should be better about reporting file views that violate this > part of the standard. We report it in a few places but clearly not > enough. > > ==rob >
