New submission from STINNER Victor <vstin...@python.org>:
The long-term goal of the PEP 554 is to run two Python interpreters in parallel. To achieve this goal, no object must be shared between two interpreters. See for example my article "Pass the Python thread state explicitly" which gives a longer rationale: https://vstinner.github.io/cpython-pass-tstate.html In bpo-38858, I modified Objects/longobject.c to have per-interpreter small integer singletons: commit 630c8df5cf126594f8c1c4579c1888ca80a29d59. This issue is about other singletons like None or Py_True which are currently shared between two interpreters. I propose to add new functions. Example for None: * Py_GetNone(): return a *borrowed* reference to the None singleton (similar to existing Py_None macro) * Py_GetNoneRef(): return a *strong* reference to the None singleton (similar to "Py_INCREF(Py_None); return Py_None;" and Py_RETURN_NONE macro) And add PyInterpreterState.none field: strong reference to the per-interpreter None object. We should do that for each singletons: * None (Py_None) * True (Py_True) * False (Py_False) * Ellipsis (Py_Ellipsis) GIL issue ========= Py_GetNone() would look like: PyObject* Py_GetNone(void) { return _PyThreadState_GET()->interp->none; } Problem: _PyThreadState_GET() returns NULL if the caller function doesn't hold the GIL. Using the Python C API when the GIL is not held is a violation of the API: it is not supported. But it worked previously. One solution is to fail with an assertion error (abort the process) in debug mode, and let Python crash in release mode. Another option is to only fail with an assertion error in debug mode in Python 3.9. In Python 3.9, Py_GetNone() would use PyGILState_GetThisThreadState() function which works even when the GIL is released. In Python 3.10, we would switch to _PyThreadState_GET() and so crash in release mode. One concrete example of such issue can be found in the multiprocessing C code, in semlock_acquire(): Py_BEGIN_ALLOW_THREADS if (timeout_obj == Py_None) { res = sem_wait(self->handle); } else { res = sem_timedwait(self->handle, &deadline); } Py_END_ALLOW_THREADS Py_None is accessed when the GIL is released. ---------- components: Interpreter Core messages: 361121 nosy: vstinner priority: normal severity: normal status: open title: [subinterpreters] Per-interpreter singletons (None, True, False, etc.) versions: Python 3.9 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue39511> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
