[issue34781] infinite waiting in multiprocessing.Pool

Change by Antonio Vázquez Blanco : -- nosy: +antonio.vazquez ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: h

[issue34781] infinite waiting in multiprocessing.Pool

Change by Myles Steinhauser : -- nosy: +myles.steinhauser ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: htt

[issue34781] infinite waiting in multiprocessing.Pool

Dongyan Li added the comment: I got the same issue with Python 3.7.2 on Windows Build 14393. Seems that the program got stuck on the `waiter.acquire()` method. -- nosy: +Dongyan Li ___ Python tracker __

[issue34781] infinite waiting in multiprocessing.Pool

Tomáš Bouda added the comment: By now I have spent several days trying to reproduce the behaviour in production environment with debugger attached. Unfortunately, no success. On the other hand yesterday the application froze, again, and colleague today experienced the problem in his script, t

[issue34781] infinite waiting in multiprocessing.Pool

Antoine Pitrou added the comment: @calimeroteknik, this doesn't seem to have anything to do with the issue at hand. Please open a separate issue with your scripts. -- ___ Python tracker ___

[issue34781] infinite waiting in multiprocessing.Pool

Change by Antoine Pitrou : Removed file: https://bugs.python.org/file47827/hang.py ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue34781] infinite waiting in multiprocessing.Pool

Change by Antoine Pitrou : Removed file: https://bugs.python.org/file47828/except-out.py ___ Python tracker ___ ___ Python-bugs-list mailing

[issue34781] infinite waiting in multiprocessing.Pool

calimeroteknik added the comment: Attaching the version that randomly raises ChildProcessError: [Errno 10] No child processes. The child process is lost in limbo if we don't sleep/print after creating it. -- Added file: https://bugs.python.org/file47828/except-out.py

[issue34781] infinite waiting in multiprocessing.Pool

calimeroteknik added the comment: A friend has found a very simple example that triggers such an issue in a very reproducible manner. Attached two versions, one where the child process mysteriously disappears in the cpython interpreter. pypy is unaffected. -- nosy: +calimeroteknik

[issue34781] infinite waiting in multiprocessing.Pool

Antoine Pitrou added the comment: If I understand correctly you have two cases: - the standalone script hangs with 3.6 on OS X - a much more involved use case hangs with 2.7 on RedHat It's possible you are hitting two different bugs. The 2.7 issue may be due to third-party packages or other

[issue34781] infinite waiting in multiprocessing.Pool

Tomáš Bouda added the comment: Oh, I should add that by decreasing number of workers to 4 or 8 the problem disappeared, at least to the extent when I wasn't able to reproduce it on any environment. -- ___ Python tracker

[issue34781] infinite waiting in multiprocessing.Pool

Tomáš Bouda added the comment: It's very difficult to reproduce. In this example to get stuck on 3.6/OSX I need to attach debugger. However, the freeze happens regardless of print/logging, even def f(): pass can get stuck. os.write() made no difference and frozen, as well, as I've just tried.

[issue34781] infinite waiting in multiprocessing.Pool

Antoine Pitrou added the comment: I couldn't reproduce with Python 3.6.5 on Ubuntu 18.04. Does it happen if you reduce logging? Or if you replace f() with: def f(i): os.write(1, "{}\n".format(i).encode()) -- ___ Python tracker

[issue34781] infinite waiting in multiprocessing.Pool

Change by Brett Cannon : -- nosy: +davin, pitrou ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail

[issue34781] infinite waiting in multiprocessing.Pool

Tomáš Bouda added the comment: After more digging, I found that the following happens: popen_fork.py -> _launch(self, process_obj) -> self.pid = os.fork() When I let process (both child and parent) print resulting pid, on freezing I can see: a) 50-times pid > 0 b) 49-times pid == 0 That mean

[issue34781] infinite waiting in multiprocessing.Pool

New submission from Tomáš Bouda : I have encountered a possible bug inside multiprocessing.Pool which behaves like race-condition while I don't believe it is a typical one. Simply put, Pool from time to time freezes. It is occasional and hard to reproduce, but e.g. unit-tests running 3/day fre