New submission from Hartmut Goebel: When using
Py_GetPath(); Py_SetPath(pypath_w); near the beginning of a program, Py_SetPath crashes with memory corruption if run on a systems with glibc's MALLOC_CHECK enabled. The reason is: Py_GetPath and Py_SetPath use different memory interfaces. * Py_GetPath() calls calculate_path(), which uses PyMem_New() to allocate the memory (see https://github.com/python/cpython/blob/v3.6.0/Modules/getpath.c#L738, assigned to `buf` first) * But Py_SetPath() uses PyMem_RawFree() to free the memory (see https://github.com/python/cpython/blob/v3.6.0/Modules/getpath.c#L830). This error DOES NOT occur on Windows, since for win32 there is a separate implementation, which uses PyMem_RawMalloc (see https://github.com/python/cpython/blob/v3.6.0/PC/getpathp.c#L378). This error also DOES NOT occur in Python <= 3.5, since PYMEM_FUNCS have been the same as PYRAW_FUNCS (see https://github.com/python/cpython/blob/v3.5.0/Objects/obmalloc.c#L148). This was changed in v3.6.0: PYMEM_FUNCS now is PYOBJ_FUNCS, see https://github.com/python/cpython/blob/v3.6.0/Objects/obmalloc.c#L159. This error only occurs when running on a system with glibc's MALLOC_CHECK enabled, which seems to be the default st least on CentOS, Fedora and ArchLinux. Example code and relevant source see below. Example stack trace ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ *** glibc detected *** ./dist/jcollect3: double free or corruption (out): 0x00007fde271ab030 *** ======= Backtrace: ========= /lib64/libc.so.6[0x37f0675e66] /lib64/libc.so.6[0x37f06789b3] /tmp/_MEIhKg3kx/libpython3.6m.so.1.0(Py_SetPath+0x15)[0x7fde2d20e825] ./dist/jcollect3[0x4043e2] ./dist/jcollect3[0x402db2] ./dist/jcollect3[0x402fb0] /lib64/libc.so.6(__libc_start_main+0xfd)[0x37f061ed5d] ./dist/jcollect3[0x401a9e] ======= Memory map: ======== How to reproduce ~~~~~~~~~~~~~~~~~~~~~~~~~ An example for this problem is the bootloader of PyInstaller (as of 2017-08-01). * Make sure you are running on a Linux distribution having glibc's MALLOC_CHECK enabled. * Set up a virtual environment (just too keep your system clean) pip install https://github.com/pyinstaller/pyinstaller/archive/develop@%7B2017-08-01%7D.zip echo 'print("Hello")' > test.py pyinstaller test.py dist/test/test # run the frozen script The relevant source of PyInstaller's bootloader is at <https://github.com/pyinstaller/pyinstaller/blob/develop@%7B2017-08-01%7D/bootloader/src/pyi_pythonlib.c#L466> ---------- components: Interpreter Core messages: 302624 nosy: htgoebel priority: normal severity: normal status: open title: Py_GetPath, Py_SetPath memory corruption due to mixed PyMem_New micex with PyMem_Raw_Free type: crash versions: Python 3.6 _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue31532> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
