2011/9/11 蓝色基因 <bluegene8...@gmail.com>:
> This is my first touch on the multiprocessing module, and I admit not
> having a deep understanding of parallel programming, forgive me if
> there's any obvious error. This is my test code:
>
> # deadlock.py
>
> import multiprocessing
>
> class MPTask:
> def __init__(self):
> self._tseq= range(10) # task sequence
> self._pool= multiprocessing.Pool(2) # process pool
> def _exe(self, num):
> return num**2
> def run(self):
> result= self._pool.map_async(self._exe, self._tseq)
> return result.get()
>
> result= MPTask().run()
> print(result)
>
> And seemingly it creates a deadlock, I have to manually kill the
> processes in the system monitor, yet it would be OK if the _exe()
> function was defined outside the MPTask class:
>
> # no_deadlock.py
> import multiprocessing
>
> def _exe(num):
> return num**2
>
> class MPTask:
> def __init__(self):
> self._tseq= range(10) # task sequence
> self._pool= multiprocessing.Pool(2) # process pool
> def run(self):
> result= self._pool.map_async(_exe, self._tseq)
> return result.get()
>
> result= MPTask().run()
> print(result)
>
> This would give the correct answer without any deadlock. My questions:
>
> 1. Why is this, where did the deadlock come from?
>
> 2. Besides, is there any material I can refer to about how to avoid
> deadlocks when using multiple processes in the program?
>
I get this exception when I run the first program:
Exception in thread Thread-1:
Traceback (most recent call last):
File "/usr/lib/python3.1/threading.py", line 516, in _bootstrap_inner
self.run()
File "/usr/lib/python3.1/threading.py", line 469, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.1/multiprocessing/pool.py", line 231, in _handle_tasks
put(task)
File "/usr/lib/python3.1/pickle.py", line 1349, in dumps
Pickler(f, protocol, fix_imports=fix_imports).dump(obj)
_pickle.PicklingError: Can't pickle <class 'method'>: attribute lookup
builtins.method failed
There is no deadlock. You are hitting this problem:
http://stackoverflow.com/questions/1816958/cant-pickle-type-instancemethod-when-using-pythons-multiprocessing-pool-map
--
regards,
kushal
--
http://mail.python.org/mailman/listinfo/python-list
