I have a C++ library (luckily with source code, so I know what's going on) to which I wrote c bindings. The library internally starts an event pthread which generates callbacks back into python. The c binding for processing these callbacks look like this (simplified for convenience):
PyGILState_STATE gstate; gstate = PyGILState_Ensure(); PyObject_CallObject(cb, arglist); PyGILState_Release(gstate); This works perfectly while the python program using this c binding is running. However, when the program terminates normally and there happens to be one last callback underway, the c bindings hang. When viewed with GDB, I can see that the callback thread callstack looks like this: futex_abstimed_wait_cancelable() _pthread_cond_wait_common() __pthread_cond_timedwait() PyCOND_TIMEDWAIT() take_gil() PyEval_RestoreThread() PyGILState_Ensure() c_bindings_callback() cpp_library_callback() and the main thread callstack, which is where the python interpreter was running, looks like this __pthread_clockjoin_ex () std::thread::join() cpp_library_static_singleton_class_destructor () __run_exit_handlers () __GI_exit () __libc_start_main () _start () in other words, the program hangs because the main thread is waiting for the event pthread to join, but that thread is stuck in a callback waiting for the GIL. What is the right way to prevent this problem from happening? Thank you in advance, Paul. P.S. I am running on Linux: ubuntu 18.04 with python 3.6.9, also reproduced with python 3.7.5, as well as ubuntu 20.04 with python 3.8.5 -- https://mail.python.org/mailman/listinfo/python-list
