[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Davin Potts added the comment: Closing this issue after having verified that the issue can no longer be reproduced on the systems mentioned (Ubuntu 10.04 or OSX). Related issues such as issue9205 have been addressed elsewhere and other possibly related issues such as issue22393 are being trac

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Changes by Paul Nasrat : -- nosy: +pnasrat ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.or

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Changes by Gökçen Eraslan : -- nosy: +Gökçen.Eraslan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Matthew Walker added the comment: Oh, and the stack trace was identical to Greg's: $ ./test.py I am process number 10378 : i = 0 [...] I am process number 10390 : i = 9 [0, 1, 4, 9, 16, 25, 36, 49, 64, 81] Exception in thread Thread-1 (most likely raised during interpreter shutdown): Traceback

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Matthew Walker added the comment: It looks to me as if this issue has already been pretty much sorted out already. Maybe all it lacks is to be officially closed, but just in case I just wanted to add that I too saw this bug (stock python 2.7, Ubuntu 10.04 64 bit). My example code was: #!/u

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Changes by Albert Strasheim : -- nosy: +Albert.Strasheim ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: Thank you for doing that footwork Greg, it means a lot to me. I'm leaning towards the patch to swallow the errors - I just wanted to ponder it just a tiny bit longer before I pull the trigger. -- ___ Python tracker

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Greg Brockman added the comment: With pool.py:272 commented out, running about 50k iterations, I saw 4 tracebacks giving an exception on pool.py:152. So this seems to imply the race does exist (i.e. that the thread is in _maintain_pool rather than time.sleep when shutdown begins). It looks

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: Talking with Brett; the fix should be as simple as keeping a reference to the debug function which we have in the imports. During interpreter shutdown, the sys.modules is iterated and each module replaced with None. Since the _handle_workers thread persists sli

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Changes by Ask Solem : -- nosy: +asksol ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/m

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: Greg - yeah. it's the same problem. -- ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubs

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Greg Brockman added the comment: Think http://www.mail-archive.com/python-l...@python.org/msg282114.html is relevant? -- ___ Python tracker ___ _

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: I'm not sure if there would still be the possibility; the thing which worries me is the debug() function vanishing on us - something not good is happening on interpreter shutdown. -- ___ Python tracker

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Greg Brockman added the comment: With the line commented out, I no longer see any exceptions. Although, if I understand what's going on, there still a (much rarer) possibility of an exception, right? I guess in the common case, the worker_handler is in the sleep when shutdown begins. But if

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: Greg, can you comment out line 272 in Lib/multiprocessing/pool.py and tell me if you can reproduce? -- ___ Python tracker ___ ___

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Greg Brockman added the comment: Yeah, I've just taken a checkout from trunk, ran './configure && make && make install', and reproduced on: - Ubuntu 10.04 32-bit - Ubuntu 9.04 32-bit -- ___ Python tracker ___

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: Correction; it can and does happen on OS/X. So, this is not a platform specific bug. -- ___ Python tracker ___ __

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: It does not seem to appear on OS/X 10.6.4 - so the only question is does this show up on Ubuntu 32bit -- ___ Python tracker ___ _

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: I can confirm with a clean ubuntu 64 install, with a clean checkout of release27 that it explodes with that exception, while the stock 2.6.5 does not. -- ___ Python tracker

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: Alright, I'm fighting ubuntu 64 bit in my vmware install right now, I'll see if I can get it up and running. -- ___ Python tracker ___ __

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Greg Brockman added the comment: > Wait - so, you are pulling svn trunk, compiling and running your test > with the built python executable? Yes. I initially observed this issue while using 10.04's Python (2.6.5), but wanted to make sure it wasn't fixed by using a newer interpreter. > I'm no

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: Wait - so, you are pulling svn trunk, compiling and running your test with the built python executable? I'm not following the "multiprocessing-from-trunk" distinction unless you're picking the module out of the tree / compiling it and then moving it into some o

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Greg Brockman added the comment: No, I'm not using the Google code backport. To be clear, I've tried testing this with two versions of multiprocessing: - multiprocessing-from-trunk (r82645): I get these exceptions with ~40% frequency - multiprocessing from Ubuntu 10.04 (version 0.70a1): No suc

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Jesse Noller added the comment: Oh, you mean the backport from google code? The person who stepped up to maintain that has not refreshed that in some time. I need to decide what to do with it long term. I'm pretty sure it's badly out of date. -- _

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Greg Brockman added the comment: That's likely a mistake on my part. I'm not observing this using the stock version of multiprocessing on my Ubuntu machine(after running O(100) times). I do, however, observe it when using either python2.7 or python2.6 with multiprocessing-from-trunk, if tha

[issue9207] multiprocessing occasionally spits out exception during shutdown (_handle_workers)

Changes by Jesse Noller : -- title: multiprocessing occasionally spits out exception during shutdown -> multiprocessing occasionally spits out exception during shutdown (_handle_workers) ___ Python tracker ___

[issue9207] multiprocessing occasionally spits out exception during shutdown

Jesse Noller added the comment: Thanks greg; so this affects 2.6 as well (not using the backport at all) -- assignee: -> jnoller nosy: +jnoller ___ Python tracker ___ __

[issue9207] multiprocessing occasionally spits out exception during shutdown

New submission from Greg Brockman : On Ubuntu 10.04, using freshly-compiled python-from-trunk (as well as multiprocessing-from-trunk), I get tracebacks from the following about 30% of the time: """ import multiprocessing, time def foo(x): time.sleep(3)