[issue28696] imap from ThreadPool hangs by an exception in a generator function
New submission from Lev Veshnyakov:
It's only in imap, in map it's ok. The following code explains the issue:
from multiprocessing.pool import ThreadPool
pool = ThreadPool(10)
def gen():
yield 1 + '1' # here is an error
try:
next((pool.imap(str, gen(
except:
# Will be catched using pool.map normally, but using pool.imap will be not.
# Instead it hangs. This is the same for ThreadPool and Pool.
print('this will not be printed because thread is hanging')
--
components: Library (Lib)
files: test.py
messages: 280833
nosy: lev-veshnyakov
priority: normal
severity: normal
status: open
title: imap from ThreadPool hangs by an exception in a generator function
type: behavior
versions: Python 3.4, Python 3.5
Added file: http://bugs.python.org/file45486/test.py
___
Python tracker
<http://bugs.python.org/issue28696>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28696] imap from ThreadPool hangs by an exception in a generator function
Lev Veshnyakov added the comment: Ubuntu 14.04 LTS, 3.13.0-83-generic, x86_64 -- ___ Python tracker <http://bugs.python.org/issue28696> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28696] imap from ThreadPool hangs by an exception in a generator function
Lev Veshnyakov added the comment: So, I've checked twice, it's presented by me on python 3.4.3, and not by 3.5.2. So I will go deaper -- ___ Python tracker <http://bugs.python.org/issue28696> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28696] imap from ThreadPool hangs by an exception in a generator function
Lev Veshnyakov added the comment: I've reproduced it on 2 different machines: - on a MacBook in Docker (debian:jessie, python 3.4.2) - on another desktop (Ubuntu 14.04.1, 3.16.0-77-generic, x86_64, python 3.4.3) -- ___ Python tracker <http://bugs.python.org/issue28696> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28696] imap from ThreadPool hangs by an exception in a generator function
Lev Veshnyakov added the comment:
It's hanging in a while loop in _handle_workers,
/usr/lib/python3.4/multiprocessingpool.py:365.
I can't figure out what is the reason.
@staticmethod
def _handle_workers(pool):
thread = threading.current_thread()
# Keep maintaining workers until the cache gets drained, unless the pool
# is terminated.
while thread._state == RUN or (pool._cache and thread._state != TERMINATE):
pool._maintain_pool()
time.sleep(0.1)
# send sentinel to stop workers
pool._taskqueue.put(None)
util.debug('worker handler exiting')
--
___
Python tracker
<http://bugs.python.org/issue28696>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28696] imap from ThreadPool hangs by an exception in a generator function
Lev Veshnyakov added the comment: Yes, I'm free to move to 3.5, now I'm seeing isn't there any problems in 3.5 according to this issue. -- ___ Python tracker <http://bugs.python.org/issue28696> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28699] Imap from ThreadPool behaves unexpectedly
New submission from Lev Veshnyakov: Consider the following code: from multiprocessing.pool import ThreadPool pool = ThreadPool(10) def gen(): yield 1 + '1' # here is an error print(list(pool.imap(str, gen( # prints [] print(list(pool.map(str, gen( # raises TypeError The difference is, that the line with imap prints an empty list, while the line with map raises an exception, as expected. Change the above snippet of code, adding additional yield statement: from multiprocessing.pool import ThreadPool pool = ThreadPool(10) def gen(): yield 1 yield 1 + '1' # here is an error print(list(pool.imap(str, gen( # raises TypeError print(list(pool.map(str, gen( # also would raise TypeError So now both map and imap will raise the exception, as expected. Therefore I suppose the behavior of imap showed in the first case is wrong. -- components: Library (Lib) messages: 280872 nosy: lev-veshnyakov priority: normal severity: normal status: open title: Imap from ThreadPool behaves unexpectedly type: behavior versions: Python 3.5 ___ Python tracker <http://bugs.python.org/issue28699> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue28699] Imap from ThreadPool behaves unexpectedly
Changes by Lev Veshnyakov : -- nosy: +davin ___ Python tracker <http://bugs.python.org/issue28699> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
