[issue13146] Writing a pyc file is not atomic

Changes by Antoine Pitrou : -- status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://m

[issue13146] Writing a pyc file is not atomic

Barry A. Warsaw added the comment: On Aug 15, 2013, at 08:25 PM, Antoine Pitrou wrote: >Barry, do you still want to keep this issue open? I don't necessarily need to. We've patched the Ubuntu version to be safe, so I guess we'll just carry that delta along until 3.4. -- _

[issue13146] Writing a pyc file is not atomic

Antoine Pitrou added the comment: Barry, do you still want to keep this issue open? -- ___ Python tracker ___ ___ Python-bugs-list mai

[issue13146] Writing a pyc file is not atomic

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue13146] Writing a pyc file is not atomic

Charles-François Natali added the comment: The workaround would be to unlink the file first, and then try to create it with O_EXCL. You have a short window where there's no file, but that shouldn't be a problem in this specific case, and it would work on Windows. As for issue #17222, well, many

[issue13146] Writing a pyc file is not atomic

Barry A. Warsaw added the comment: On May 20, 2013, at 09:57 PM, Charles-François Natali wrote: >IIRC, os.rename() will fail on Windows if the target file already exists. >That's why os.replace() was added. Ah, that's probably a more serious blocker for adding it to upstream Python. Not so for

[issue13146] Writing a pyc file is not atomic

Antoine Pitrou added the comment: > IIRC, os.rename() will fail on Windows if the target file already > exists. Good point. -- ___ Python tracker ___ ___

[issue13146] Writing a pyc file is not atomic

Charles-François Natali added the comment: IIRC, os.rename() will fail on Windows if the target file already exists. That's why os.replace() was added. -- ___ Python tracker ___

[issue13146] Writing a pyc file is not atomic

Barry A. Warsaw added the comment: On May 20, 2013, at 09:52 PM, Antoine Pitrou wrote: >Some people already complained about this change. I'm not sure it's fit for a >bugfix release. http://bugs.python.org/issue17222 Yeah, but that's a crazy use case. :) >Besides, you can just also make py_co

[issue13146] Writing a pyc file is not atomic

Antoine Pitrou added the comment: > I'm re-opening this because I'd like to get RM pronouncement on > applying a patch to 2.7, 3.2, and 3.3 to make py_compile.py atomically > rename its pyc/pyo file. Some people already complained about this change. I'm not sure it's fit for a bugfix release. ht

[issue13146] Writing a pyc file is not atomic

Barry A. Warsaw added the comment: Oh btw, if Georg and Benjamin deny this for the stable releases, I'll very likely patch the Ubuntu versions anyway. -- ___ Python tracker ___

[issue13146] Writing a pyc file is not atomic

Barry A. Warsaw added the comment: I'm re-opening this because I'd like to get RM pronouncement on applying a patch to 2.7, 3.2, and 3.3 to make py_compile.py atomically rename its pyc/pyo file. Attached is a patch for 2.7 based on importlib's approach in 3.4. It should be easy enough to por

[issue13146] Writing a pyc file is not atomic

Barry A. Warsaw added the comment: Proposed patch for 2.7 -- Added file: http://bugs.python.org/file30324/13146-2.7.patch ___ Python tracker ___ _

[issue13146] Writing a pyc file is not atomic

Changes by Barry A. Warsaw : -- nosy: +doko ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.

[issue13146] Writing a pyc file is not atomic

Changes by Christian Heimes : -- nosy: +christian.heimes ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http:/

[issue13146] Writing a pyc file is not atomic

Changes by Barry A. Warsaw : -- nosy: +barry ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python

[issue13146] Writing a pyc file is not atomic

Brett Cannon added the comment: And thanks for doing this, Antoine! One less thing on my never-ending todo list. =) On Mon, Oct 17, 2011 at 10:35, Antoine Pitrou wrote: > > Antoine Pitrou added the comment: > > Should be fixed now. Thanks for the reviews! > > -- > resolution:  -> fixe

[issue13146] Writing a pyc file is not atomic

Antoine Pitrou added the comment: Should be fixed now. Thanks for the reviews! -- resolution: -> fixed stage: patch review -> committed/rejected status: open -> closed ___ Python tracker _

[issue13146] Writing a pyc file is not atomic

Roundup Robot added the comment: New changeset c16063765d3a by Antoine Pitrou in branch 'default': Issue #13146: Writing a pyc file is now atomic under POSIX. http://hg.python.org/cpython/rev/c16063765d3a -- nosy: +python-dev ___ Python tracker

[issue13146] Writing a pyc file is not atomic

Antoine Pitrou added the comment: Ok, here is a new patch using O_EXCL. Also, since import.c is quite different in 3.2, I'm not sure I will bother backporting. -- versions: -Python 3.2 Added file: http://bugs.python.org/file23396/importrename3.patch __

[issue13146] Writing a pyc file is not atomic

STINNER Victor added the comment: > with atomic_create(, 'b') as f: See issues #8604 and #8828. -- ___ Python tracker ___ ___ Python

[issue13146] Writing a pyc file is not atomic

Charles-François Natali added the comment: > Or perhaps append the PID to the name of the temp file ? > (easier done in Python than in C :-)) I don't really like appending PIDs to generate file names: - if you have multiple processes at the same time, they'll all write their own file which will

[issue13146] Writing a pyc file is not atomic

Antoine Pitrou added the comment: > > This new patch also fixes importlib. > > """ > path_tmp = path + '.tmp' > with _io.FileIO(path_tmp, 'wb') as file: > file.write(data) > _os.rename(path_tmp, path) > """ > > I don't know ex

[issue13146] Writing a pyc file is not atomic

Charles-François Natali added the comment: > Here is a patch for import.c. Looks good to me. > This new patch also fixes importlib. """ path_tmp = path + '.tmp' with _io.FileIO(path_tmp, 'wb') as file: file.write(data) _os.re

[issue13146] Writing a pyc file is not atomic

Antoine Pitrou added the comment: > So if a process replaces the PYC file whereas another is reading the > PYC, the reader may read corrupted data? No, this is the whole point of the patch. -- ___ Python tracker

[issue13146] Writing a pyc file is not atomic

STINNER Victor added the comment: So if a process replaces the PYC file whereas another is reading the PYC, the reader may read corrupted data? The ideal fix is maybe to use a file lock? -- nosy: +haypo ___ Python tracker

[issue13146] Writing a pyc file is not atomic

Antoine Pitrou added the comment: This new patch also fixes importlib. -- stage: -> patch review versions: -Python 2.7 Added file: http://bugs.python.org/file23378/importrename2.patch ___ Python tracker

[issue13146] Writing a pyc file is not atomic

Antoine Pitrou added the comment: Here is a patch for import.c. -- keywords: +patch Added file: http://bugs.python.org/file23375/importrename.patch ___ Python tracker ___ __

[issue13146] Writing a pyc file is not atomic

New submission from Antoine Pitrou : One of the buildbots recently showed the following failure: == ERROR: test_rapid_restart (test.test_multiprocessing.WithProcessesTestManagerRestart) -