On Mon, Nov 23, 2009 at 11:17:45AM +0100, Christoph Rackwitz wrote: > >If I may ask a slightly different question: you've got periods of I/O > >and periods of computation. Have you evaluated collective I/O? > > I thought about it and I know a way to make it happen too, but I put > that on the "to do" pile for possible improvements later on, after > I'd have gotten the asynchronous I/O working. My file format > contains a struct followed by two matrices (same dimensions). Right > now, I write the header via rank 0 and then each rank writes one > stripe for each matrix, resulting in two Requests pending. I gather > that I'd need to construct one or two more data types for > split-collective I/O to be applicable, i.e., so the whole write > happens in one call.
If the other processors need header data, perhaps rank 0 can broadcast it to everyone else? You won't get any overlap with split collectives in any ROMIO-based MPI-IO: they have always been implemented in a blocking fashion. You're on the right track: describe the I/O for the two matrices with one datatype, then on rank zero use that datatype as one of the datatype for a struct type. It's definitely a different way of thinking about I/O than the async model. Is it OK to mention MPICH2 on this list? I did prototype some MPI extensions that allowed ROMIO to do true async I/O (at least as far as the underlying operating system supports it). If you really need to experiment with async I/O, I'd love to hear your experiences. ==rob -- Rob Latham Mathematics and Computer Science Division Argonne National Lab, IL USA
