[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: I merged more changes in bpo-39877 which made possible to finally fix this issue. -- components: +Interpreter Core resolution: -> fixed stage: patch review -> resolved status: open -> closed versions: +Python 3.9 -Python 3.4

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: New changeset 9ad58acbe8b90b4d0f2d2e139e38bb5aa32b7fb6 by Victor Stinner in branch 'master': bpo-19466: Py_Finalize() clears daemon threads earlier (GH-18848) https://github.com/python/cpython/commit/9ad58acbe8b90b4d0f2d2e139e38bb5aa32b7fb6 --

[issue19466] Clear state of threads earlier in Python shutdown

Change by STINNER Victor : -- pull_requests: +18212 pull_request: https://github.com/python/cpython/pull/18854 ___ Python tracker ___ __

[issue19466] Clear state of threads earlier in Python shutdown

Change by STINNER Victor : Added file: https://bugs.python.org/file48961/asyncio_gc.py ___ Python tracker ___ ___ Python-bugs-list mailing l

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: asyncio_gc.py: script to reproduce bpo-20526 (copied from there), but ported to Python 3.9 (replace asyncio.async with asyncio.ensure_future). Sadly, Python with PR 18848 does crash when running asyncio_gc.py if I press CTRL+c twice: * A thread does crash i

[issue19466] Clear state of threads earlier in Python shutdown

Change by STINNER Victor : -- pull_requests: +18205 stage: resolved -> patch review pull_request: https://github.com/python/cpython/pull/18848 ___ Python tracker ___ __

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: The change has been reverted in 2014. I reopen the issue since using daemon thraeds became safer in Python 3.9. As soon as a daemon thread attempts to take the GIL, it does exit immediately. With the commit eb4e2ae2b8486e8ee4249218b95d94a9f0cc513e (bpo-39877

[issue19466] Clear state of threads earlier in Python shutdown

Roundup Robot added the comment: New changeset 9ce58a73b6b5 by Victor Stinner in branch '3.4': Issue #20526, #19466: Revert changes of issue #19466 which introduces a http://hg.python.org/cpython/rev/9ce58a73b6b5 -- ___ Python tracker

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: I reverted the changes because it introduced regressions. -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___

[issue19466] Clear state of threads earlier in Python shutdown

Roundup Robot added the comment: New changeset 1166b3321012 by Victor Stinner in branch 'default': Issue #20526, #19466: Revert changes of issue #19466 which introduces a http://hg.python.org/cpython/rev/1166b3321012 -- ___ Python tracker

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: A new bug was found because of changes of this issue: #20526. IMO it is safer to revert changes of this issue in Python 3.4, it's too late to analyze such tricky bug. See #20526 for the discussion. -- resolution: fixed -> status: closed -> open _

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: > Could someone review my message http://bugs.python.org/msg206028? I replied with questions and you didn't reply :-) -- ___ Python tracker ___

[issue19466] Clear state of threads earlier in Python shutdown

Charles-François Natali added the comment: > I didn't see test_threading failing anymore recently, so I'm closing the issue. Thanks. Hm... Could someone review my message http://bugs.python.org/msg206028? Because AFAICT, this will lead to random crashes with daemon threads. -- __

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: I didn't see test_threading failing anymore recently, so I'm closing the issue. Thanks. -- resolution: -> fixed status: open -> closed ___ Python tracker

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: "It's actually fairly easy to write a short crasher launching a deamon thread reading from e.g. /dev/zero in loop, with the main thread exiting right in the middle." I'm unable to write such crasher, can you please give an example? Are you able to crash Python

[issue19466] Clear state of threads earlier in Python shutdown

Charles-François Natali added the comment: Hum... Correct me if I'm wrong, but destroying the thread state of daemon threads while they're running is really a bad idea in fact: for example, if warnings are now emitted for unclosed file objects, this means that the file object, and all associated

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: I'm unable to reproduce the test_4_daemon_threads failure, can anyone try on Windows? -- ___ Python tracker ___ ___

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: The fix raised a new issue: #19565. I also saw a crash on Windows which is probably related: http://buildbot.python.org/all/builders/x86 Windows Server 2003 [SB] 3.x/builds/1717/steps/test/logs/stdio ==

[issue19466] Clear state of threads earlier in Python shutdown

Roundup Robot added the comment: New changeset 10a8e676b87b by Victor Stinner in branch 'default': Issue #19466: Fix typo. Patch written by Vajrasky Kok. http://hg.python.org/cpython/rev/10a8e676b87b -- ___ Python tracker

[issue19466] Clear state of threads earlier in Python shutdown

Vajrasky Kok added the comment: Typo: s/immediatly/immediately. I attached the patch. -- nosy: +vajrasky Added file: http://bugs.python.org/file32586/fix_typo_19466.patch ___ Python tracker __

[issue19466] Clear state of threads earlier in Python shutdown

Roundup Robot added the comment: New changeset c2a13acd5e2b by Victor Stinner in branch 'default': Close #19466: Clear the frames of daemon threads earlier during the Python http://hg.python.org/cpython/rev/c2a13acd5e2b -- nosy: +python-dev resolution: -> fixed stage: -> committed/reje

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: neologix made a review on rietveld: http://bugs.python.org/review/19466/#ps9818 There was an issue in finalize_threads-2.patch: the test didn't pass in release mode. I fixed it by adding -Wd to the command line option. I also replaced threading.Lock with threa

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: > You are clearing the thread state of the currently executing > thread, which doesn't sound right. Oh, I didn't realize that PyThreadState_Clear() clears also Python thread locals. Here is a new patch without PyThreadState_Clear(tstate) and with two unit test

[issue19466] Clear state of threads earlier in Python shutdown

Antoine Pitrou added the comment: I am referring to the following part of your patch: +/* Clear the state of the current thread */ +PyThreadState_Clear(tstate); You are clearing the thread state of the currently executing thread, which doesn't sound right. -- nosy: +neologix

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: 2013/11/4 Serhiy Storchaka : > Is it possible to write a test for this behavior? It is possible to test it manually using warn_shutdown.py attached to #19442. The warnings module may be used to test the change. -- __

[issue19466] Clear state of threads earlier in Python shutdown

STINNER Victor added the comment: 2013/11/4 Antoine Pitrou : > I'm afraid clearing thread states is a bit too brutal. What if some > destructor relies on contents of the thread states (e.g. thread locals)? When Py_Finalize() is called, only one Python thread hold the GIL. After _Py_Finalizing=t

[issue19466] Clear state of threads earlier in Python shutdown

Antoine Pitrou added the comment: I'm afraid clearing thread states is a bit too brutal. What if some destructor relies on contents of the thread states (e.g. thread locals)? -- ___ Python tracker

[issue19466] Clear state of threads earlier in Python shutdown

Serhiy Storchaka added the comment: Is it possible to write a test for this behavior? -- ___ Python tracker ___ ___ Python-bugs-list m

[issue19466] Clear state of threads earlier in Python shutdown

Changes by STINNER Victor : -- nosy: +serhiy.storchaka ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://

[issue19466] Clear state of threads earlier in Python shutdown

Changes by Arfrever Frehtes Taifersar Arahesis : -- nosy: +Arfrever ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscri

[issue19466] Clear state of threads earlier in Python shutdown

New submission from STINNER Victor: Each Python thread holds references to objects, in its current frame for example. At Python shutdown, clearing threads state happens very late: the import machinery is already dead, types are finalized, etc. If a thread has an object with a destructor, bad t