New submission from Ronan Lamy <ronan.l...@gmail.com>:
There seems to be a race condition in importlib._bootstrap._ModuleLock that can
cause a deadlock. The sequence of operations is as follows:
* Thread 1 calls lock.acquire()
* Thread 1 sets itself as lock.owner and begins importing the module
* Thread 2 calls lock.acquire() and waits for lock.lock
* Thread 2 gets lock.lock
* Thread 1 calls lock.acquire() again, due to a nested import
* Thread 1 sets itself as blocking on lock: _blocking_on[tid1] = lock
* Thread 2 enters lock.has_deadlock()
* Thread 2 busy-waits forever in has_deadlock() because lock.owner == tid1 and
_blocking_on[tid1] == lock
* Thread 1 waits forever for lock.lock since thread 2 owns it
The issue was found in pypy3 but it also affects all the recent CPython
versions I tried.
I can reliably reproduce the issue by adding an artificial delay to
_ModuleLock.has_deadlock(), e.g. with this patch:
diff --git a/Lib/test/test_import/__init__.py b/Lib/test/test_import/__init__.py
index f167c84..7f7188e 100644
--- a/Lib/test/test_import/__init__.py
+++ b/Lib/test/test_import/__init__.py
@@ -435,10 +435,15 @@ class ImportTests(unittest.TestCase):
os.does_not_exist
def test_concurrency(self):
+ def delay_has_deadlock(frame, event, arg):
+ if event == 'call' and frame.f_code.co_name == 'has_deadlock':
+ time.sleep(0.2)
+
sys.path.insert(0, os.path.join(os.path.dirname(__file__), 'data'))
try:
exc = None
def run():
+ sys.settrace(delay_has_deadlock)
event.wait()
try:
import package
----------
components: Library (Lib)
messages: 351647
nosy: Ronan.Lamy
priority: normal
severity: normal
status: open
title: Import deadlock detection causes deadlock
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue38091>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
