Inada Naoki <songofaca...@gmail.com> added the comment:
Memory allocation pattern is: alloc 24 # float alloc 24 alloc 24 alloc 64 # temporary tuple alloc 72 <repeat> free 64 # free temporary tuples free 64 free 64 <repeat> This cause some sort of fragmentation. Some pools in arenas are unused. This prevents pymalloc to return arenas to OS. (Note that pymalloc manages memory as arena (256KiB) > pool (4KiB) > blocks (requested sizes <= 512). pymalloc can return the memory to OS only when arena is clean) But this is not too bad because many pools is free. Any allocation which size < 512 can reuse the free pools. If you run some code after unpickle, the pools will be reused efficiently. (In case of very bad fragmentation, many pools are dirty: some blocks in pools are used while many blocks in the pool is free. So only same size alloc request can use the pool.) There are two approach to fix this problem. 1. Investigate why temporary tuple is not freed until last stage of unpickle. 2. When there are too many free pools, return some by MADV_FREE or MADV_DONTNEED. I think (1) should be considered first. But (2) is more general solution. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <https://bugs.python.org/issue36694> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
