New submission from Stanislaw Izaak Pitucha:
This is something that happened once and I cannot reproduce anymore. In an
IPython session I've done the following (see session below), which resulted in
pool.map raising its internal exceptions in cases where it shouldn't. Then it
worked again. (see in/out 29)
Running exactly the same lines again did not result in the same behaviour. I
tried multiple times in the same session as well as new sessions. Looks like a
weird race / corruption.
I'm on Ubuntu's '3.4.3 (default, Mar 26 2015, 22:03:40) \n[GCC 4.9.2]'. Running
with 2 virtualised cores (vmware). 64-bit system.
In [21]: import multiprocessing
In [22]: p=multiprocessing.Pool()
... (unrelated, checked p.map? and other helps)
In [26]: class A(object):
def a(self, i):
print("boo", i, multiprocessing.current_process())
....:
In [27]: obj = A()
In [28]: p.map(obj.a, [1,2,3,4])
Process ForkPoolWorker-1:
Process ForkPoolWorker-2:
Traceback (most recent call last):
Traceback (most recent call last):
File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
self.run()
File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
self.run()
File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.4/multiprocessing/pool.py", line 108, in worker
task = get()
File "/usr/lib/python3.4/multiprocessing/pool.py", line 108, in worker
task = get()
File "/usr/lib/python3.4/multiprocessing/queues.py", line 357, in get
return ForkingPickler.loads(res)
AttributeError: Can't get attribute 'A' on <module '__main__'>
File "/usr/lib/python3.4/multiprocessing/queues.py", line 357, in get
return ForkingPickler.loads(res)
AttributeError: Can't get attribute 'A' on <module '__main__'>
boo 3 <ForkProcess(ForkPoolWorker-3, started daemon)>
boo 4 <ForkProcess(ForkPoolWorker-3, started daemon)>
^CProcess ForkPoolWorker-4:
Process ForkPoolWorker-3:
Traceback (most recent call last):
Traceback (most recent call last):
File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
self.run()
File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.4/multiprocessing/process.py", line 254, in _bootstrap
self.run()
File "/usr/lib/python3.4/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.4/multiprocessing/pool.py", line 108, in worker
task = get()
File "/usr/lib/python3.4/multiprocessing/pool.py", line 108, in worker
task = get()
File "/usr/lib/python3.4/multiprocessing/queues.py", line 354, in get
with self._rlock:
File "/usr/lib/python3.4/multiprocessing/queues.py", line 355, in get
res = self._reader.recv_bytes()
File "/usr/lib/python3.4/multiprocessing/synchronize.py", line 96, in
__enter__
return self._semlock.__enter__()
KeyboardInterrupt
File "/usr/lib/python3.4/multiprocessing/connection.py", line 216, in
recv_bytes
buf = self._recv_bytes(maxlength)
File "/usr/lib/python3.4/multiprocessing/connection.py", line 416, in
_recv_bytes
buf = self._recv(4)
File "/usr/lib/python3.4/multiprocessing/connection.py", line 383, in _recv
chunk = read(handle, remaining)
KeyboardInterrupt
---------------------------------------------------------------------------
KeyboardInterrupt Traceback (most recent call last)
<ipython-input-28-3be6fbcc359b> in <module>()
----> 1 p.map(obj.a, [1,2,3,4])
/usr/lib/python3.4/multiprocessing/pool.py in map(self, func, iterable,
chunksize)
258 in a list that is returned.
259 '''
--> 260 return self._map_async(func, iterable, mapstar, chunksize).get()
261
262 def starmap(self, func, iterable, chunksize=None):
/usr/lib/python3.4/multiprocessing/pool.py in get(self, timeout)
591
592 def get(self, timeout=None):
--> 593 self.wait(timeout)
594 if not self.ready():
595 raise TimeoutError
/usr/lib/python3.4/multiprocessing/pool.py in wait(self, timeout)
588
589 def wait(self, timeout=None):
--> 590 self._event.wait(timeout)
591
592 def get(self, timeout=None):
/usr/lib/python3.4/threading.py in wait(self, timeout)
551 signaled = self._flag
552 if not signaled:
--> 553 signaled = self._cond.wait(timeout)
554 return signaled
555 finally:
/usr/lib/python3.4/threading.py in wait(self, timeout)
288 try: # restore state no matter what (e.g., KeyboardInterrupt)
289 if timeout is None:
--> 290 waiter.acquire()
291 gotit = True
292 else:
KeyboardInterrupt:
In [29]: p.map(obj.a, [1,2,3,4])
boo 1 <ForkProcess(ForkPoolWorker-5, started daemon)>
boo 2 <ForkProcess(ForkPoolWorker-6, started daemon)>
boo 3 <ForkProcess(ForkPoolWorker-5, started daemon)>
boo 4 <ForkProcess(ForkPoolWorker-6, started daemon)>
Out[29]: [None, None, None, None]
----------
components: Library (Lib)
messages: 250362
nosy: Stanislaw Izaak Pitucha
priority: normal
severity: normal
status: open
title: Possible race condition in Pool
type: behavior
versions: Python 3.4
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue25053>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
