New submission from Josh Rosenberg: Unclear if this is just unclear docs, or incorrect behavior.
Per this Stack Overflow question ( https://stackoverflow.com/questions/36191447/why-doesnt-the-daemon-program-exit-without-join ), you get some rather odd behavior when you have both daemon and non-daemon child processes. In the case described, the following steps occur: 1. A daemon Process is launched which prints a message, waits two seconds, then prints a second message 2. The main process sleeps one second 3. A non-daemon process is launched which behaves the same as the daemon process, but sleeps six seconds before the second message. 4. The main process completes The expected behavior (to my mind and the questioner on SO) is that since there is a non-daemon process running, the "process family" should stay alive until the non-daemon process finishes, which gives the daemon process time to wake up and print its second message (five seconds before the non-daemon process wakes to finish its "work"). But in fact, the atexit function used for cleanup in multiprocessing first calls .terminate() on all daemon children before join-ing all children. So the moment the main process completes, it immediately terminates the daemon child, even though the "process family" is still alive. This seems counter-intuitive; in the threading case, which multiprocessing is supposed to emulate, all non-daemon threads are equivalent, so no daemon threads are cleaned until the last non-daemon thread exits. To match the threading behavior, it seems like the cleanup code should first join all the non-daemon children, then terminate the daemon children, then join the daemon children. This would change the code here ( https://hg.python.org/cpython/file/3.5/Lib/multiprocessing/util.py#l303 ) from: for p in active_children(): if p.daemon: info('calling terminate() for daemon %s', p.name) p._popen.terminate() for p in active_children(): info('calling join() for process %s', p.name) p.join() to: # Wait on non-daemons first for p in active_children(): info('calling join() for process %s', p.name) if not p.daemon: p.join() # Terminate and clean up daemons now that non-daemons done for p in active_children(): if p.daemon: info('calling terminate() for daemon %s', p.name) p._popen.terminate() info('calling join() for process %s', p.name) p.join() I've attached repro code to demonstrate; using multiprocessing, the daemon never prints its exiting message, while switching to multiprocessing.dummy (backed by threading) correctly prints the exit message. ---------- components: Library (Lib) files: testmpdaemon.py messages: 262324 nosy: josh.r priority: normal severity: normal status: open title: multiprocessing behavior combining daemon with non-daemon children inconsistent with threading type: behavior versions: Python 2.7, Python 3.5, Python 3.6 Added file: http://bugs.python.org/file42265/testmpdaemon.py _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue26633> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
