New submission from Will Starms <vilhel...@gmail.com>: Pool's join function currently (3.6.3) lacks a timeout, which can cause the managing thread to sleep indefinitely when a pool worker hangs or starts misbehaving. Adding a timeout allows the owning thread to attempt a join and, after the timeout, return to other tasks, such as monitoring worker health.
In my specific situation, I have a Pool running a task on a large set of files. If any single task fails, the whole operation is ruined and the pool should be terminated. A task can communicate with the main thread through error_callback, but if the thread has already called join, it can't check until join returns, after the Pool has finished all processing. Attached is an incredibly simple patch to the current (3.6) cpython implementation that emulates threading.thread.join's behavior. ---------- components: Library (Lib) files: cpython_timeout.patch keywords: patch messages: 304350 nosy: Will Starms priority: normal severity: normal status: open title: Add a timeout to multiprocessing's Pool.join type: enhancement versions: Python 3.6 Added file: https://bugs.python.org/file47219/cpython_timeout.patch _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31782> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
