New submission from Jesse Noller <jnol...@gmail.com>: I have example code to show this. It creates a system-wide memory leak on Linux/Unix (present until the next reboot), unless the last statement in the target of mp.Process ensures a manual clean up of the globals.
The problem is line 353 in multiprocessing/forking.py. The function exit() is defined as os._exit on Linux and ExitProcess on Windows, none of which allows normal clean up. >>> help(os._exit) Help on built-in function _exit in module nt: _exit(...) _exit(status) Exit to the system with specified status, without normal exit processing. The problem is fixed if line 353 in forking.py is changed from exit(exitcode) to sys.exit(exitcode) Test run without bugfix: G:\DEVELO~1\SHARED~2>python test.py open handle to 569f439b24e24fc8a547b81932616066 [[ 0. 0. 0. 0.] [ 0. 0. 0. 0.]] open handle to 0582d4b161c546f582c1c96e7bd0c39d open handle to 569f439b24e24fc8a547b81932616066 modified array closed handle to 569f439b24e24fc8a547b81932616066 [[ 1. 1. 1. 0.] [ 1. 1. 1. 0.]] closed handle to 569f439b24e24fc8a547b81932616066 You can see here that opening and closing of handles are unmatched. This is on Windows, where the kernel ensures the clean-up, so it may not matter. But on Unix this would have created a permament (system wide) memory leak! What is happening here is globals not being cleaned up due to the use of os._exit instead of sys.exit. Test run with bugfix: G:\DEVELO~1\SHARED~2>python test.py open handle to 930778d27b414253bc329f2b70adaa05 [[ 0. 0. 0. 0.] [ 0. 0. 0. 0.]] open handle to 3f6cebf8c5de413685bb770d02ae9666 open handle to 930778d27b414253bc329f2b70adaa05 modified array closed handle to 930778d27b414253bc329f2b70adaa05 closed handle to 3f6cebf8c5de413685bb770d02ae9666 [[ 1. 1. 1. 0.] [ 1. 1. 1. 0.]] closed handle to 930778d27b414253bc329f2b70adaa05 Now all allocations and deallocations are matched. Regards, Sturla Molden ---------- files: test.zip messages: 91332 nosy: jnoller severity: normal status: open title: Potential memory leak in multiprocessing Added file: http://bugs.python.org/file14660/test.zip _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue6653> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
