[issue17969] multiprocessing crash on exit

Changes by Kristján Valur Jónsson : -- resolution: -> wont fix ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue17969] multiprocessing crash on exit

Kristján Valur Jónsson added the comment: Closing this as won-t fix. Exiting with running threads is a can of worms. -- status: open -> closed ___ Python tracker ___ ___

[issue17969] multiprocessing crash on exit

Kristján Valur Jónsson added the comment: Richard, reading the multiprocessing.py code, it appears that your suggestion of closign and joining the Pool() object should also do the trick. Doing that will certainly also fix this particular case. I'll implement that in our local application cod

[issue17969] multiprocessing crash on exit

Kristján Valur Jónsson added the comment: Richard, I'll review implement your change. It is a bit tricky to test this, since I can only tell after a few days if the particular (rare) problem has been fixed. The crash is a rare one but consistently happens with some probability we use multipr

[issue17969] multiprocessing crash on exit

Charles-François Natali added the comment: > Catching regressions is what we have the regression tests for. If it is not > in caught by the regression tests, then it is not a regression, by definition. You do realize this sentence doesn't make sense, do you? -- __

[issue17969] multiprocessing crash on exit

Antoine Pitrou added the comment: > Catching regressions is what we have the regression tests for. If it > is not in caught by the regression tests, then it is not a > regression, by definition. Call it what you want :-) The bottom line is that we'll release a 2.7.5 soon because of breakage int

[issue17969] multiprocessing crash on exit

Richard Oudkerk added the comment: Kristjan, could you confirm whether joining the pool explicitly before shutdown (in the way I suggested earlier) fixes the problem. I think it should -- at shutdown you won't switch to a thread if it has already been joined. -- _

[issue17969] multiprocessing crash on exit

Kristján Valur Jónsson added the comment: Catching regressions is what we have the regression tests for. If it is not in caught by the regression tests, then it is not a regression, by definition. Bug fix mode is for fixing bugs, IMHO. Yes, I have patched my local version. The reason I am here

[issue17969] multiprocessing crash on exit

Richard Oudkerk added the comment: > We can consider two options then: > 1) A multiprocessing specific fix. Removing this handle close gil > release (which is superfluous, since these calls aren't blocking in any > real sense) will certainly remove _this_ instance of the crash. An > alternat

[issue17969] multiprocessing crash on exit

Antoine Pitrou added the comment: Le lundi 13 mai 2013 à 16:14 +, Kristján Valur Jónsson a écrit : > I don't see the point of having 2.7 in bug fix mode if we can't fix > bugs. Delicate bug fixes may entail regressions, and we've had enough of them in 2.7.4. You've already patched your own P

[issue17969] multiprocessing crash on exit

Kristján Valur Jónsson added the comment: Well, knowing that they crash less in 3.3 doesn't really fix the problem now, does it? We can consider two options then: 1) A multiprocessing specific fix. Removing this handle close gil release (which is superfluous, since these calls aren't blocking

[issue17969] multiprocessing crash on exit

Antoine Pitrou added the comment: Yes, I think it's too delicate to change in 2.7 right now. As Charles-François said, daemon threads should be much less crashy in 3.3. -- ___ Python tracker __

[issue17969] multiprocessing crash on exit

Charles-François Natali added the comment: > Also, even though an IO call _can_ block, that doesn't mean that > we _must_ release the gil for the duration. Yes, otherwise some people will complain when the whole interpreter is stuck while a socket/NFS file handle/whatever is shutdown. This pro

[issue17969] multiprocessing crash on exit

Kristján Valur Jónsson added the comment: I think that socket.close() is the exception rather than the rule here. What kind of handle is this? It can't be a socket, since that would require closesocket. Also, even though an IO call _can_ block, that doesn't mean that we _must_ release the g

[issue17969] multiprocessing crash on exit

Richard Oudkerk added the comment: > In general, deallocators should have no side effects, I think. > Releaseing the GIL is certainly a side effect. Notice that socket and file objects also release the GIL when being deallocated. At least for sockets close() can block (e.g. if you you use th

[issue17969] multiprocessing crash on exit

Changes by Richard Oudkerk : -- nosy: +sbt ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.o

[issue17969] multiprocessing crash on exit

New submission from Kristján Valur Jónsson: We have observed this crash with some frequency when running our compilation scripts using multiprocessing.Pool() By analysing the crashes, this is what is happening: 1) The Pool has a "daemon" thread managing the pool. 2) the worker is asleep, waitin