Paul Rubin wrote: > Is there any way to get Python to release memory back to the > C allocator? I'm currently running a script that goes through > the following steps: > > 1) Creates a very large number of Python objects to produce > a relatively small data structure that sits in a C extension. > The Python objects consume quite a bit of memory. > > 2) Releases all the Python objects. > > 3) Invokes a function of said C extension for further > processing. This step needs as much memory as possible. > ... > > I happen to have the code for the C library in question, but I > don't think this is the way to go in general. If there's a way to > get Python to give memory back to the C allocator I can avoid > touching the library at all. > > A cave-man approach might be to fork a new process after step 1, pass > the small data structure to it, and have the old process exit > (releasing all its memory back to the OS). The new process then > carries out the remaining steps.
I think I see what you're doing, but fork() after step 1 will create a child process with the same memory allocated. I think it would make more sense to do step 1 in a subprocess. Use the subprocess module or one of the older popen()s to create a process that builds the target object, pickles it and pipes it back to the main process, then exits. -- --Bryan -- http://mail.python.org/mailman/listinfo/python-list
