On Jun 17, 8:51 pm, Squzer Crawler <[EMAIL PROTECTED]> wrote:
> i am developing distributed environment in my college using Python. I
> am using therads in client for downloading wepages. Even though i am
> reusing the thread, memory usage get increased. I don know why.? I am
> using BerkelyDB for URLQueue, BeautifulShop for Parsing the webpages.
Isn't the increased memory resulted from storing the already
processed pages?
Look first at all places where your code instantiates new
objects - and make sure you don't keep references to such objects that
are not needed anymore.
Also, reusing threads has nothing to do with saving memory - but
with saving on thread creation time, if I understand your problem
description.
--
http://mail.python.org/mailman/listinfo/python-list
