Charles-François Natali <neolo...@free.fr> added the comment:
> Or perhaps append the PID to the name of the temp file ?
> (easier done in Python than in C :-))
I don't really like appending PIDs to generate file names:
- if you have multiple processes at the same time, they'll all write
their own file which will end up being replaced by the last one to
perform the move, whereas with O_EXCL, they'll see immediately that
another instance is writing it (the overhead is negligible with such
small files, but maybe not so much when creating the file requires a
certain amout of work)
- if processes crash at the wrong time, you can end up with a flurry
of <filename>.<PID>
- the last one is even more insidious and unlikely, but here it goes:
the PID is unique only on a given machine: if you have, for example, a
network file system shared between multiple hosts, then you can have a
PID collision, whereas O_EXCL is safe (O_EXCL doesn't work on NFSv2,
but nowadays every OS implements it correctly on NFSv3)
O_EXCL is really what POSIX offers to solve this (and it's also what
import.c does).
>
>> Also, as a side note, I'm wondering whether this type of check:
>> """
>> if not sys.platform.startswith('win'):
>> # On POSIX-like platforms, renaming is atomic
>> """
>>
>> couldn't be rewritten as
>> """
>> if os.name == 'posix':
>> # On POSIX-like platforms, renaming is atomic
>> """
>
> No, because os.py is not available to importlib (which must be
> bootstrappable early). See the _bootstrap.py header for information
> about what is available; this is also why we use FileIO instead of
> open().
OK. So is the O_EXCL approach possible? Would something like
_io.open(_os.open(path, _os.O_CREATE|os.O_EXCL...), 'wb')
work?
Also, since this can be quite tricky and redundant, how about adding a
framework to do this kind of thing to the standard library?
Something like
with atomic_create(<final path>, 'b') as f:
f.write(<data>)
where atomic_create would be a context manager that would make `f`
point to a temporary file (open with O_EXCL :-), and do the rename at
the end. It could also accept an option to ensure durability (i.e.
call fsync() on the file and on the parent directory). Note that it
probably wouldn't help here, since we only have access to a really
limited part of the library :-)
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue13146>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
