New submission from Giovanni Pizzi <gio....@gmail.com>:
Hello, when creating deleting (with `os.remove`/`os.unlink`) a file and opening it in a different process at (almost) the same time, on Mac OS X relatively often I get a file that is empty instead of either a `FileNotFoundError` exception, or an open handle with the original content, i.e. at least one of the two operations (unlinking or opening) seems to be non-atomic. (With empty I mean that if after I open the file (mode 'rb'), then `fhandle.read()` returns me b''. More in particular, after quite some debugging I noticed that what happens is that if I stat the file descriptor, `st_ino` is zero. This can reproduced very easily. I set up a GitHub repository here: https://github.com/giovannipizzi/concurrent-delete-race-problem with simple examples and tests (and both a Python implementation and a C implementation). There are also GitHub Actions that run on Mac, Ubuntu and Windows, and apparently the problem exists only on Mac (and happens on all versions). For completeness I attach also here a tar.gz with the two very short python files that just need be run in parallel - in my Mac OS X (macOS 10.14.6, 15" 2016 MacBook Pro, python 3.6) I get the error essentially at every run. Important: while much harder to reproduce, I can get the same error and behaviour also with the (I think equivalent) C code. Therefore, this seems to be more a problem with the Mac OS standard libraries? My first question is: has anybody ever seen this problem? It seems to me quite easy to reproduce, but I'm surprised that I couldn't find any reference in the internet even after searching for various hours (maybe I used the wrong keywords?) Second question: should this be reported directly to Apple? Third question: Even if this is a bug, would it make sense to implement a patch in python that raises some exception if st_ino is zero when opening a file? Or am I simplifying too much and in some conditions st_ino=0 is valid on some types of mounted filesystems? Is there some other way to have a more atomic behaviour for this race condition in python? Thanks a lot! ---------- components: IO, macOS files: concurrency-tests.tar.gz messages: 373606 nosy: Giovanni Pizzi, ned.deily, ronaldoussoren priority: normal severity: normal status: open title: Race conditions when opening and deleting a file on Mac OS X versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8 Added file: https://bugs.python.org/file49315/concurrency-tests.tar.gz _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue41291> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
