Bugs item #613222, was opened at 2002-09-23 14:27 Message generated for change (Settings changed) made by gbrandl You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=613222&group_id=5470
Please note that this message will contain a full copy of the comment thread, including the initial issue submission, for this request, not just the latest update. Category: Python Interpreter Core >Group: Python 2.4 Status: Open Resolution: None Priority: 3 Submitted By: Neal Norwitz (nnorwitz) Assigned to: Guido van Rossum (gvanrossum) Summary: memory leaks when importing posix module Initial Comment: The attached program which calls Py_Initialize/Py_Finalize in a loop demonstrates a program which grows quite quickly. This bug effects 2.2.1+ and 2.3. valgrind reports that the memory is still reachable, but it seems like a memory leak and the process definitely grows. Compile the program with libpython, and run (./mem-test 100000). Make sure it can import site (which imports posix module). While the program is running, do a ps and watch it grow. If import site fails, the process will not grow. site.py can be as simple as import posix. I believe the problem is related to PyStructSequence_Fields for statfs. But haven't completely diagnosed the problem. As I learn more, I will add comments. To simply importing or not importing site, mem-test takes an optional 2nd argument which will enable/disable loading of site.py. ./mem-test 100000 1 will prevent import site. I hope this is understandable, even though the description wasn't clear. ---------------------------------------------------------------------- Comment By: Facundo Batista (facundobatista) Date: 2005-03-21 18:04 Message: Logged In: YES user_id=752496 Changed to 2.3 group. ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2002-12-26 15:16 Message: Logged In: YES user_id=6380 I expect I will have no time to look into this further any time soon. But if you have fixes that clearly fix leaks, please check them in! (Shouldn't this be in the 2.3 group?) ---------------------------------------------------------------------- Comment By: Neal Norwitz (nnorwitz) Date: 2002-09-25 00:37 Message: Logged In: YES user_id=33168 I'm mostly stuck. I haven't solved the problem, but I do have some more information. Attached is a patch to Object/structseq.c which fixes some problems if memory allocation fails. Also, I changed a PyInt_FromLong(1) to Py_True for __safe_for_unpickling__. Py_True could also be used in compile.c:: symtable_load_symbols replacing the variable: implicit. Some fields which I believe are leaking from the PyTypeObject are: tp_members, tp_dict, and tp_bases. However, there are more leaks. I think all the remaining leaks come from PyType_Ready(). For example, from add_operators(), PyDescr_NewWrapper(). I thought DECREFing tp_dict would free these, but it didn't seem to have any effect. Part of the problem is how should these values be cleaned up. Putting cleanup in PyStructSequence_InitType would guarantee the problem is fixed for all structseqs, but that doesn't seem like a good idea. This would assume the PyTypeObject passed in is initialized. This is true for static variables, but not for any other variables. If there's a new API for cleanup, all the users of structseq will need to use it. Perhaps, this is only an issue in the core. I don't know about extension modules. Finally, I suspect there may be more leaks outside of posix too. These seem to mostly come from _Py_ReadyTypes() called in pythonrun.c::Py_Initialize(). ---------------------------------------------------------------------- Comment By: Guido van Rossum (gvanrossum) Date: 2002-09-23 16:32 Message: Logged In: YES user_id=6380 This is a good use of your time. Thanks for looking into this! Assign back to me when you have a fix for review or a stumbling block. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=105470&aid=613222&group_id=5470 _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
