New submission from Eric Gorr <ericg...@gmail.com>:
I have the following code:
import multiprocessing
from multiprocessing import Pool, Manager
import time
import random
def worker_function( index, messages ):
print( "%d: Entered" % index )
time.sleep( random.randint( 3, 15 ) )
messages.put( "From: %d" % index )
print( "%d: Exited" % index )
manager = Manager()
messages = manager.Queue()
with Pool( processes = None ) as pool:
for x in range( 30 ):
pool.apply_async( worker_function, [ x, messages ] )
pool.close()
pool.join()
It does not terminate -- all entered messages are printed, but not all exited
messages are printed.
If I remove all the code related to the Manager and Queue, it will terminate
properly with all messages printed.
If I assign processes explicitly, I can continue to increase the number
assigned to processes and have it continue to work until I reach a value of 20
or 21. > 20, it fails all of the time. With a value == 20 it fails some of the
time. With a value of < 20, it always succeeds.
multiprocessing.cpu_count() returns 24 for my MacPro.
----------
components: Library (Lib), macOS
messages: 312718
nosy: Eric Gorr, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Multiprocessing worker functions not terminating with a large number of
processes and a manager
versions: Python 3.6
_______________________________________
Python tracker <rep...@bugs.python.org>
<https://bugs.python.org/issue32937>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
