juj added the comment: While the test case can be 'fixed' by changing the code to use "if __name__ == '__main__'", and I'm ok to do it in my code to work around the problem, I would argue the following:
1) calling this not a bug (or solving it only at documentation level) does not at all feel correct to reflect the situation, since the provided test case silently fails and does the unexpected. If atexit() does not work at all when invoked as a result of importing from multiprocessing.Pool.map(), then at minimum it would be better that calling atexit() in such a scenario should throw an exception "not available", rather than silently discarding the operation. 2) Why couldn't the atexit handlers be executed even on Windows when the multiprocessing processes quit, even if special code is required in python multiprocessing libraries to handle it? The explanation you are giving sounds like a lazy excuse. There should not be any technical obstacle why the cleanup handlers could not be tracked and honored here? 3) Saying that this should not be working like the (existing) documentation implies, is not at all obvious to the reader. I could not find it documented that processes that exit from multiprocessing would be somehow special, and the note that you pasted does is not in any way obvious to connect to this case, since a) I was not using signals, b) there was no internal error occurring, and c) I was not calling os._exit(). The documentation does not reflect that it is undefined whether atexit() handlers are executed or not when multiprocessing is used. 4) I would even argue that it is a bug that there is different cross platform observable behavior in terms of multiprocessing and script importing, but that is probably a different topic. Overall, leaving this as a silent failure, instead of raising an exception, nor implementing the support on Windows, does not feel mature, since it leaves a hole of C/C++ style of undefined behavior in the libraries. For maturity, I would recommend something to be done, in the descending order of preference: I) Fix multiprocessing importing on windows so that it is not a special case compared to other OSes. II) If the above is not possible, fix the atexit() handlers so that they are executed when the processes quit on Windows. III) If the above is not possible, make the atexit() function raise an exception if invoked from a script that has been spawned from multiprocessing, when it is known at atexit() call time that the script was spawned a as a result of multiprocessing, and the atexit() handlers will never be run. If none of those are really not possible due to real technical reasons, then as a last resort, explicitly document both in the docs for atexit() and the docs for multiprocessing that the atexit() handlers are not executed if called on Windows when these two are used in conjunction. Disregarding these kind of silent failure behavior especially when cross-platformness is involved with a shrug and a NotABug label is not a good practice! ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23489> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
