New submission from STINNER Victor: Each Python thread holds references to objects, in its current frame for example. At Python shutdown, clearing threads state happens very late: the import machinery is already dead, types are finalized, etc. If a thread has an object with a destructor, bad things may happen.
For example, when open files are destroyed, a warning is emitted. The problem is that you cannot emits warnings because the warnings module has been unloaded, and so you miss warnings. See the issue #19442 for this specific case. It is possible to clear the Python threads earlier since Python 3.2, because Python threads will now exit cleanly when they try to acquire the GIL: see PyEval_RestoreThread(). The value of the tstate pointer is used in PyEval_RestoreThread(), but the pointer is not dereferenced (the content of a Python state is not read). So it is even possible to free the memory of the threads state (not only to clear the threads state). Attached patch implements destroy the all threads except the current thread, and clear the state of the current thread. The patch calls also the garbage collector before flushing stdout and stderr, and disable signal handlers just before PyImport_Cleanup(). The main idea is to reorder functions like this: - clear state of all threads to release strong references -> may call destructores - force a garbage collector to release more references -> may call destructores - flush stdout and stderr -> write pending warnings and any other buffered messages - disable signal handler -> only at the end so previous steps can still be interrupted by CTRL+c The side effect of the patch is that the destructor of destroyed objects are now called, especially for daemon threads. If you try the warn_shutdown.py script attached to #19442, you now get the warning with the patch! As a result, test_threading.test_4_daemon_threads() is also modified by my patch to ignore warnings :-) If I understood correctly, the patch only has an impact on daemon threads, the behaviour of classic threads is unchanged. ---------- files: finalize_threads.patch keywords: patch messages: 201860 nosy: haypo, pitrou priority: normal severity: normal status: open title: Clear state of threads earlier in Python shutdown versions: Python 3.4 Added file: http://bugs.python.org/file32443/finalize_threads.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue19466> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
