Agreed that it is not good (and I am recoding some programs to avoid this), but (and here life gets interesting) is this a "bug" in Lustre?
On Tue, Feb 2, 2010 at 5:59 PM, David Singleton <david.single...@anu.edu.au> wrote: > > But its a very bad idea on a "coherent", "POSIX" filesystem like Lustre. > Locks have to bounce around between the nodes for every write. This can > be VERY slow (even for trivial amounts of "logging" IO) and thrash the > filesystem for other users. So, yes, at our site, we include this sort > of "parallel IO" on our list of disallowed behaviour. Not a good practice > to adopt in general. > > David > > On 02/03/2010 10:40 AM, Laurence Marks wrote: >> >> I know it's wrong, but I don't think it is forbidden.... (which I >> guess is what you are saying). >> >> On Tue, Feb 2, 2010 at 5:31 PM, Jeff Squyres<jsquy...@cisco.com> wrote: >>> >>> +1 on Nick's responses. >>> >>> AFAIK, if you don't mind getting garbage in the output file, it should be >>> fine to do. Specifically: it should not cause OS issues (crash, reboot, >>> corrupted filesystem, etc.) to do this -- but the file contents will likely >>> be garbage. >>> >>> That being said, this situation likely falls into the "Doc, it hurts when >>> I do this..." category. Meaning: you know it's wrong, so you probably >>> shouldn't be doing it anyway. :-) >>> >>> >>> On Feb 2, 2010, at 4:50 PM, Nicolas Bock wrote: >>> >>>> Hi Laurence, >>>> >>>> I don't know whether it's as bad as a deadly sin, but for us parallel >>>> writes are a huge problem and we get complete garbage in the file. Take a >>>> look at: >>>> >>>> Implementing MPI-IO Atomic Mode and Shared File Pointers Using MPI >>>> One-Sided Communication, Robert Latham,Robert Ross, Rajeev Thakur, >>>> International Journal of High Performance Computing Applications, 21, 132 >>>> (2007). >>>> >>>> They describe an implemenation of a "mutex" like object in MPI. If you >>>> protect writes to the file with an exclusive lock you can serialize the >>>> writes and make use of NFS's close to open cache coherence. >>>> >>>> nick >>>> >>>> >>>> On Tue, Feb 2, 2010 at 08:27, Laurence Marks<l-ma...@northwestern.edu> >>>> wrote: >>>> I have a question concerning having many processors in a mpi job all >>>> write to the same file -- not using mpi calls but with standard >>>> fortran I/O. I know that this can lead to consistency issues, but it >>>> can also lead to OS issues with some flavors of nfs. >>>> >>>> At least in fortran, there is nothing "wrong" with doing this. My >>>> question is whether this is "One of the Seven Deadly Sins" of mpi >>>> programming, or just frowned on. (That is, it should be OK even if it >>>> leads to nonsense files, and not lead to OS issues.) If it is a sin, I >>>> would appreciate a link to where this is spelt out in some "official" >>>> document or similar. >>>> >>>> -- >>>> Laurence Marks >>>> Department of Materials Science and Engineering >>>> MSE Rm 2036 Cook Hall >>>> 2220 N Campus Drive >>>> Northwestern University >>>> Evanston, IL 60208, USA >>>> Tel: (847) 491-3996 Fax: (847) 491-7820 >>>> email: L-marks at northwestern dot edu >>>> Web: www.numis.northwestern.edu >>>> Chair, Commission on Electron Crystallography of IUCR >>>> www.numis.northwestern.edu/ >>>> Electron crystallography is the branch of science that uses electron >>>> scattering and imaging to study the structure of matter. >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>>> >>>> _______________________________________________ >>>> users mailing list >>>> us...@open-mpi.org >>>> http://www.open-mpi.org/mailman/listinfo.cgi/users >>> >>> >>> -- >>> Jeff Squyres >>> jsquy...@cisco.com >>> >>> > _______________________________________________ > users mailing list > us...@open-mpi.org > http://www.open-mpi.org/mailman/listinfo.cgi/users > -- Laurence Marks Department of Materials Science and Engineering MSE Rm 2036 Cook Hall 2220 N Campus Drive Northwestern University Evanston, IL 60208, USA Tel: (847) 491-3996 Fax: (847) 491-7820 email: L-marks at northwestern dot edu Web: www.numis.northwestern.edu Chair, Commission on Electron Crystallography of IUCR www.numis.northwestern.edu/ Electron crystallography is the branch of science that uses electron scattering and imaging to study the structure of matter.
