Its definitely not a bug in Lustre - its an essential part of any
"coherent" cluster filesystem (CXFS, QFS, GFS, GPFS, ....). The
whole point is that some people actually want to have meaningful
(non-garbage) files accessed using properly managed parallel IO
techniques. The locking is necessary in this case. You would see
the same issues on any of those filesystems.
David
On 02/03/2010 11:37 AM, Laurence Marks wrote:
Agreed that it is not good (and I am recoding some programs to avoid
this), but (and here life gets interesting) is this a "bug" in Lustre?
On Tue, Feb 2, 2010 at 5:59 PM, David Singleton
<david.single...@anu.edu.au> wrote:
But its a very bad idea on a "coherent", "POSIX" filesystem like Lustre.
Locks have to bounce around between the nodes for every write. This can
be VERY slow (even for trivial amounts of "logging" IO) and thrash the
filesystem for other users. So, yes, at our site, we include this sort
of "parallel IO" on our list of disallowed behaviour. Not a good practice
to adopt in general.
David
On 02/03/2010 10:40 AM, Laurence Marks wrote:
I know it's wrong, but I don't think it is forbidden.... (which I
guess is what you are saying).
On Tue, Feb 2, 2010 at 5:31 PM, Jeff Squyres<jsquy...@cisco.com> wrote:
+1 on Nick's responses.
AFAIK, if you don't mind getting garbage in the output file, it should be
fine to do. Specifically: it should not cause OS issues (crash, reboot,
corrupted filesystem, etc.) to do this -- but the file contents will likely
be garbage.
That being said, this situation likely falls into the "Doc, it hurts when
I do this..." category. Meaning: you know it's wrong, so you probably
shouldn't be doing it anyway. :-)
On Feb 2, 2010, at 4:50 PM, Nicolas Bock wrote:
Hi Laurence,
I don't know whether it's as bad as a deadly sin, but for us parallel
writes are a huge problem and we get complete garbage in the file. Take a
look at:
Implementing MPI-IO Atomic Mode and Shared File Pointers Using MPI
One-Sided Communication, Robert Latham,Robert Ross, Rajeev Thakur,
International Journal of High Performance Computing Applications, 21, 132
(2007).
They describe an implemenation of a "mutex" like object in MPI. If you
protect writes to the file with an exclusive lock you can serialize the
writes and make use of NFS's close to open cache coherence.
nick
On Tue, Feb 2, 2010 at 08:27, Laurence Marks<l-ma...@northwestern.edu>
wrote:
I have a question concerning having many processors in a mpi job all
write to the same file -- not using mpi calls but with standard
fortran I/O. I know that this can lead to consistency issues, but it
can also lead to OS issues with some flavors of nfs.
At least in fortran, there is nothing "wrong" with doing this. My
question is whether this is "One of the Seven Deadly Sins" of mpi
programming, or just frowned on. (That is, it should be OK even if it
leads to nonsense files, and not lead to OS issues.) If it is a sin, I
would appreciate a link to where this is spelt out in some "official"
document or similar.
--
Laurence Marks
Department of Materials Science and Engineering
MSE Rm 2036 Cook Hall
2220 N Campus Drive
Northwestern University
Evanston, IL 60208, USA
Tel: (847) 491-3996 Fax: (847) 491-7820
email: L-marks at northwestern dot edu
Web: www.numis.northwestern.edu
Chair, Commission on Electron Crystallography of IUCR
www.numis.northwestern.edu/
Electron crystallography is the branch of science that uses electron
scattering and imaging to study the structure of matter.
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
--
Jeff Squyres
jsquy...@cisco.com
_______________________________________________
users mailing list
us...@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users
