STINNER Victor added the comment: I added print(id(package)) in Lib/test/test_import/data/package/__init__.py: when the bug occurs on my Windows VM, I see two print with two different identifiers. Moreover, when the bug occurs, first id(sys.modules['package']) is the first identifier, and then package is replaced with the new 'package' package in sys.modules (new identifier).
I don't understand something: sys.modules isn't supposed to be protected by a lock? I checked if _installed_safely.__enter__() if the _imp is held: it isn't held when _bootstrap._setup() is called, nor when _imp._install_external_importers() is called. But even if I patch these two functions to acquire the lock, Python startup fails on importing a codec module, because the _imp lock is not held in _installed_safely.__enter__(). So maybe sys.modules dict is supposed to be protected by a different lock, maybe indirectly? I hack _bootstrap._find_and_load_unlocked() to sleep 10 ms if name == 'package'. With this change, test_concurrency() fails immediatly on my Linux box. * Apply attached importlib_sleep.patch (written for the current master branch) * Recompile: make regen-importlib && make * Run test_concurrency() just once: ./python -m test -m test_concurrency -v test_import * You get the bug By the way, importlib_sleep.patch changes test_concurrency() to run the test exactly once (it changes to loop to a single iteration, instead of 10). Is it normal that a sleep makes import failing? Imports are not supposed to be atomic? -- I tried to similar hack before the commit b4baacee1adc06edbe30ac7574d17a8cd168e2e0: * add a sleep in importlib * copy test_concurrency() from master * Run test_concurrency() * The test pass While I'm not sure of my hack, it *seems* like sys.modules was somehow atomic before this commit? ---------- keywords: +patch Added file: http://bugs.python.org/file47000/importlib_sleep.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue30891> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
