On 3/18/2011 7:54 PM, Jason Grout wrote:
Right; thanks. Let me rephrase my questions:
1. Why is important that the multiprocessing Pool worker processors
have daemon=True (I think this is the same as asking: why is it
important that they be terminated with terminate() rather than join() )?
2. Is it safe for us to reset a Pool worker process to have
daemon=False before starting a subprocess from that worker, like in
the code from the original message?
Thanks,
Jason
Jason,
I just happen to be dealing with a project that uses multiprocessing.
What I have learned is this...
If a child thread (pool worker) is not set to daemon (daemon=False), if
for some reason the parent thread terminates either normally or
abnormally and the worker thread has not completed its task, the child
thread will terminate by throwing all sorts of nasty errors. However,
in daemon mode, multiprocessing will terminate the child thread
'cleanly'. That's not to say that the worker has a chance to complete
its work or shut itself down. Multiprocessing will absorb the
exceptions and not pass them along.
You may terminate a child thread using join(). That is probably the
safest way to do it. terminate() will just kill the worker thread
immediately without any regard to whether or not it has completed its
tasks. I believe multiprocessing uses terminate() as well to kill a
daemon thread if the parent thread disappears.
join() will, however, block until the task has competed and returned.
If you want to continue doing work in the parent thread while the child
thread is off doing its thing, then another means of syncing up the
parent and children threads is needed.
As for allowing children threads to spawn off children of its own using
subprocess runs the risk of creating a little army of zombie
'grandchildren' if either the parent or child threads terminate before
the subprocess completes and returns.
Hope that helps....
John
--
http://mail.python.org/mailman/listinfo/python-list
