On Fri, Oct 6, 2017 at 8:06 AM, Fetchinson . via Python-list <python-list@python.org> wrote: > Hi folks, > > I have a rather simple program which cycles through a bunch of files, > does some operation on them, and then quits. There are 500 files > involved and each operation takes about 5-10 MB of memory. As you'll > see I tried to make every attempt at removing everything at the end of > each cycle so that memory consumption doesn't grow as the for loop > progresses, but it still does. > > import os > > for f in os.listdir( '.' ): > > x = [ ] > > for ( i, line ) in enumerate( open( f ) ): > > import mystuff > x.append( mystuff.expensive_stuff( line ) ) > del mystuff > > import mystuff > mystuff.some_more_expensive_stuff( x ) > del mystuff > del x > > > What can be the reason? I understand that mystuff might be leaky, but > if I delete it, doesn't that mean that whatever memory was allocated > is freed? Similary x is deleted so that can't possibly make the memory > consumption go up.
You're not actually deleting anything. When you say "del x", all you're doing is removing the *name* x. Especially, deleting an imported module basically does nothing; it's a complete waste of time. Modules are kept in their own special cache. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
