kaifeng <cafe...@gmail.com> added the comment:
I applied your patch to Python 3.2, also I added a function call to
'malloc_trim' via ctypes, as you can see in issue11849_test2.py.
In fact I have a daemon written in Python 2.5, parsing an XML of size 10+ MB
every 5 minutes, after 16+ hours running, the program finally exhausted 4 GB
memory and died. I simplified the logic of the daemon and found ElementTree
eats too much memory. There comes the attached test script.
BTW, after utilize lxml instead of ElementTree, such phenomenon of increasing
memory usage disappeared.
$ ltrace -e malloc_trim python3.2 Issue11849_test2.py
--- SIGCHLD (Child exited) ---
--- SIGCHLD (Child exited) ---
*** Python 3.2.0 final
--- SIGCHLD (Child exited) ---
--- PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
--- SIGCHLD (Child exited) ---
0 13708 pts/1 S+ 0:00 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:00 1 1316 11055 6440 0.6 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
1 13708 pts/1 S+ 0:00 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:03 1 1316 53155 47332 4.5 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
2 13708 pts/1 S+ 0:00 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:06 1 1316 91055 85204 8.2 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
3 13708 pts/1 S+ 0:01 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:10 1 1316 128947 124212 11.9 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
4 13708 pts/1 S+ 0:01 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:13 1 1316 166807 162280 15.6 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
5 13708 pts/1 S+ 0:01 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:16 1 1316 204483 198808 19.2 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
6 13708 pts/1 S+ 0:02 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:20 1 1316 242379 236672 22.8 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
7 13708 pts/1 S+ 0:02 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:23 1 1316 284383 277508 26.8 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
8 13708 pts/1 S+ 0:03 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:27 1 1316 318191 312436 30.1 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
9 13708 pts/1 S+ 0:03 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:29 1 1316 360199 353272 34.1 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
END 13708 pts/1 S+ 0:03 1 65 1742 636 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:34 1 1316 393975 388164 37.4 python3.2
Issue11849_test2.py
malloc_trim(0, 0, 0x818480a, 0x81a0114, 0xbfb6c940)
= 1
--- SIGCHLD (Child exited) ---
GC 13708 pts/1 S+ 0:03 1 65 1742 648 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:35 1 1316 351871 347480 33.5 python3.2
Issue11849_test2.py
--- SIGCHLD (Child exited) ---
*** 13708 pts/1 S+ 0:03 1 65 1742 648 0.0 ltrace -e
malloc_trim python3.2 Issue11849_test2.py
13709 pts/1 S+ 0:35 1 1316 351871 347480 33.5 python3.2
Issue11849_test2.py
+++ exited (status 0) +++
----------
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue11849>
_______________________________________
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
