New submission from Thomas Jackson: If a KeyboardInterrupt is received while a worker process is grabbing an item off of the queue that worker process dies with an uncaught exception. This means that the ProcessPool now has lost a process, and currently has no mechanism to recover from dead processes. This is especially noticeable if the CallItem is relatively large (as the call_queue.get() includes all the pickle time).
A simple fix is to have the worker process not do anything with the keyboard interrupt-- since it would have no idea what to do. This cannot be implemented with a regular try/except-- as the item will be partially pulled off of the queue and lost. My proposed fix is to disable the SIGINT handler in the worker process while getting items off of the queue. An alternate approach is to actually change multiprocessing.Queue.get() to leave the item on the queue if it is interrupted with a keyboard interrupt to this is to catch the KeyboardInterrupt and simply continue on-- then we can rely on the caller to do the cleanup.This cannot be done by simply Proposed patch attached ---------- files: worker_ignore_interrupt.patch keywords: patch messages: 256703 nosy: jacksontj priority: normal severity: normal status: open title: ProcessPoolExecutor deadlock on KeyboardInterrupt type: behavior versions: Python 2.7, Python 3.2, Python 3.3, Python 3.4, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file41358/worker_ignore_interrupt.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue25908> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
