Charles-Francois Natali <neolo...@free.fr> added the comment: This is definitely a malloc bug. Test with default malloc on a Debian box:
cf@neobox:~/cpython$ ./python ../issue11849_test.py *** Python 3.3.0 alpha --- PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND 0 3778 pts/2 S+ 0:00 1 1790 8245 7024 0.5 ./python ../issue11849_test.py 1 3778 pts/2 S+ 0:17 1 1790 61937 60404 4.6 ./python ../issue11849_test.py 2 3778 pts/2 S+ 0:35 1 1790 110841 108300 8.3 ./python ../issue11849_test.py 3 3778 pts/2 S+ 0:53 1 1790 159885 158540 12.2 ./python ../issue11849_test.py 4 3778 pts/2 S+ 1:10 1 1790 209369 206724 15.9 ./python ../issue11849_test.py 5 3778 pts/2 S+ 1:28 1 1790 258505 255956 19.7 ./python ../issue11849_test.py 6 3778 pts/2 S+ 1:46 1 1790 307669 304964 23.5 ./python ../issue11849_test.py 7 3778 pts/2 S+ 2:02 1 1790 360705 356952 27.5 ./python ../issue11849_test.py 8 3778 pts/2 S+ 2:21 1 1790 405529 404172 31.2 ./python ../issue11849_test.py 9 3778 pts/2 S+ 2:37 1 1790 458789 456128 35.2 ./python ../issue11849_test.py END 3778 pts/2 S+ 3:00 1 1790 504189 501624 38.7 ./python ../issue11849_test.py GC 3778 pts/2 S+ 3:01 1 1790 454689 453476 35.0 ./python ../issue11849_test.py *** 3778 pts/2 S+ 3:01 1 1790 454689 453480 35.0 ./python ../issue11849_test.py [56426 refs] The heap is not trimmed, even after GC collection. Now, using a smaller mmap threshold so that malloc uses mmap instead of brk: cf@neobox:~/cpython$ MALLOC_MMAP_THRESHOLD_=1024 ./python ../issue11849_test.py *** Python 3.3.0 alpha --- PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND 0 3843 pts/2 S+ 0:00 1 1790 8353 7036 0.5 ./python ../issue11849_test.py 1 3843 pts/2 S+ 0:17 1 1790 62593 59240 4.5 ./python ../issue11849_test.py 2 3843 pts/2 S+ 0:35 1 1790 112321 108304 8.3 ./python ../issue11849_test.py 3 3843 pts/2 S+ 0:53 1 1790 162313 157372 12.1 ./python ../issue11849_test.py 4 3843 pts/2 S+ 1:11 1 1790 212057 206456 15.9 ./python ../issue11849_test.py 5 3843 pts/2 S+ 1:29 1 1790 261749 255484 19.7 ./python ../issue11849_test.py 6 3843 pts/2 S+ 1:47 1 1790 311669 304484 23.5 ./python ../issue11849_test.py 7 3843 pts/2 S+ 2:03 1 1790 365485 356488 27.5 ./python ../issue11849_test.py 8 3843 pts/2 S+ 2:22 1 1790 411341 402568 31.1 ./python ../issue11849_test.py 9 3843 pts/2 S+ 2:38 1 1790 465141 454552 35.1 ./python ../issue11849_test.py END 3843 pts/2 S+ 3:02 1 1790 67173 63892 4.9 ./python ../issue11849_test.py GC 3843 pts/2 S+ 3:03 1 1790 9925 8664 0.6 ./python ../issue11849_test.py *** 3843 pts/2 S+ 3:03 1 1790 9925 8668 0.6 ./python ../issue11849_test.py [56428 refs] Just to be sure, with ptmalloc3 malloc implementation: cf@neobox:~/cpython$ LD_PRELOAD=../ptmalloc3/libptmalloc3.so ./python ../issue11849_test.py *** Python 3.3.0 alpha --- PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND 0 3898 pts/2 S+ 0:00 1 1790 8369 7136 0.5 ./python ../issue11849_test.py 1 3898 pts/2 S+ 0:17 1 1790 62825 60264 4.6 ./python ../issue11849_test.py 2 3898 pts/2 S+ 0:34 1 1790 112641 110176 8.5 ./python ../issue11849_test.py 3 3898 pts/2 S+ 0:52 1 1790 162689 160048 12.3 ./python ../issue11849_test.py 4 3898 pts/2 S+ 1:09 1 1790 212285 209732 16.2 ./python ../issue11849_test.py 5 3898 pts/2 S+ 1:27 1 1790 261881 259460 20.0 ./python ../issue11849_test.py 6 3898 pts/2 S+ 1:45 1 1790 311929 309332 23.9 ./python ../issue11849_test.py 7 3898 pts/2 S+ 2:01 1 1790 365625 362004 27.9 ./python ../issue11849_test.py 8 3898 pts/2 S+ 2:19 1 1790 411445 408812 31.5 ./python ../issue11849_test.py 9 3898 pts/2 S+ 2:35 1 1790 465205 461536 35.6 ./python ../issue11849_test.py END 3898 pts/2 S+ 2:58 1 1790 72141 69688 5.3 ./python ../issue11849_test.py GC 3898 pts/2 S+ 2:59 1 1790 15001 13748 1.0 ./python ../issue11849_test.py *** 3898 pts/2 S+ 2:59 1 1790 15001 13752 1.0 ./python ../issue11849_test.py [56428 refs] So the problem is really that glibc/eglibc malloc implementations don't automatically trim memory upon free (this happens if you're only allocating/deallocating small chunks < 64B that come from fastbins, but that's not the case here). By the way, I noticed that dictionnaries are never allocated through pymalloc, since a new dictionnary takes more than 256B... ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue11849> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
