Matthias Troffaes <matthias.troff...@gmail.com> added the comment: To aid the discussion, I attach another quick patch which reports the time spent on PyXXX_ClearFreeList calls during highest generation garbage collection (including gc.collect()). For simplicity, the timer uses clock() so the resolution is quite limited (appears to be 10ms on my machine) and I don't claim that this is the best way of measuring execution speed, but at least it gives some indication.
The patch also gives an indication at how frequently the highest generation is collected. Below is the result of the patch on the py3k-freelist_test.py test script, on my machine (again, debug build). For reference, I've measured the total time spent by the script as well, with the time command. Summarizing, (30+70+30+30+50+70+20+110)/48420.0 = 0.0085 = 0.85% of time is spent on freeing the freelists, in my test. Another way to look at the data is that it roughly takes 10ms for each 100MB allocated (at least for the types of data in the script). Floats seem to be an exception and take at least twice as long (not sure why). Keep in mind that the test merely allocates and deallocates memory, without doing much else, so it isn't a typical Python application. $ time ./python py3k-freelist_test.py Memory used (begin): 121Mb memtest 2000000 int ==================== Memory used (peak): 297Mb cleared free lists in 0 clock ticks (0.000000ms) Memory used (end): 121Mb Unfreed memory: 0Mb memtest 2000000 str ==================== Memory used (peak): 455Mb cleared free lists in 30000 clock ticks (30.000000ms) Memory used (end): 123Mb Unfreed memory: 2Mb memtest 2000000 float ==================== Memory used (peak): 236Mb cleared free lists in 70000 clock ticks (70.000000ms) Memory used (end): 127Mb Unfreed memory: 6Mb memtest 2000000 int ==================== Memory used (peak): 312Mb cleared free lists in 0 clock ticks (0.000000ms) Memory used (end): 123Mb Unfreed memory: 2Mb memtest 2000000 Test ==================== cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) Memory used (peak): 372Mb cleared free lists in 30000 clock ticks (30.000000ms) Memory used (end): 123Mb Unfreed memory: 2Mb memtest 2000000 Test2 ==================== cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) Memory used (peak): 361Mb cleared free lists in 30000 clock ticks (30.000000ms) Memory used (end): 123Mb Unfreed memory: 2Mb memtest 2000000 _tuple ==================== Memory used (peak): 529Mb cleared free lists in 50000 clock ticks (50.000000ms) Memory used (end): 123Mb Unfreed memory: 2Mb memtest 2000000 _set ==================== cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) Memory used (peak): 846Mb cleared free lists in 70000 clock ticks (70.000000ms) Memory used (end): 123Mb Unfreed memory: 2Mb memtest 200000 _dict ==================== Memory used (peak): 233Mb cleared free lists in 20000 clock ticks (20.000000ms) Memory used (end): 123Mb Unfreed memory: 2Mb memtest 2000000 Test3 ==================== cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) cleared free lists in 0 clock ticks (0.000000ms) Memory used (peak): 999Mb cleared free lists in 110000 clock ticks (110.000000ms) Memory used (end): 123Mb Unfreed memory: 2Mb cleared free lists in 0 clock ticks (0.000000ms) [40763 refs] real 0m50.921s user 0m48.420s sys 0m2.330s ---------- Added file: http://bugs.python.org/file14737/py3k-clearfreelist-time_gc_collect.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue6695> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
