[EMAIL PROTECTED] wrote:
> Using the same file with cElementTree took me 217 Mb, with no
> unreachable object.
> For me it's not a good behavior, it's not a good way to let the system
> swap this unused memory instead of freeing it.
> I think it's a really good idea to have a memory pool for performance
> reason, but why is there no 'free block' limit ?
> Python is a really really good language that can do many things in a
> clear, easier and performance way I think. It has always feet all my
> needs. But I can't imagine there is no good solution for that problem,
> by limiting the free block pool size or best, letting the user specify
> this limit and even better, letting the user completely freeing it
> (with also the limit manual specification)
>
> Like:
> import pool
> pool.free()
> pool.limit(size in megabytes)
>
> Why not letting the user choosing that, why not giving the user more
> flexibility ?
Because its not easy, and its an unusual edge case that hasn't attracted
developer effort (the PyMalloc change for 2.5 was contributed by someone
who desperately needed it, not a core Python developer; it was also a
non-trivial effort to get right).
You should also appreciate something about PyMalloc: it only handles
allocation requests of 256 bytes or smaller, and this limitation is part
of PyMalloc's design.
If most of your allocations are >256 bytes, you're at the mercy of the
platform malloc and heap fragmentation can be a killer. This is probably
why the getlines() approach mentioned would appear to relinquish (most
of) the memory: the list was probably comprised mostly of PyMalloc
allocations.
I haven't checked, but cElementTree may internally not be using PyMalloc
anyway, as the package is stated to be usable back to Python 1.5 - long
before the current allocation management came into effect. In which
case, you're at the mercy of the platform malloc... The pure Python
ElementTree might play more your way, at a performance cost.
--
-------------------------------------------------------------------------
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: [EMAIL PROTECTED] (pref) | Snail: PO Box 370
[EMAIL PROTECTED] (alt) | Belconnen ACT 2616
Web: http://www.andymac.org/ | Australia
--
http://mail.python.org/mailman/listinfo/python-list
