Robby Dermody wrote:
> In the diagrams above, one can see the night-day separation clearly. At
> night, the memory usage growth seemed to all but stop, but with the
> increased call volume of the day, it started shooting off again. When I
> first started gathering this data, I was hoping for a logarithmic curve,
> but at least after 48 hours, it looks like the usage increase is almost
> linear. (Although logarithmic may still be the case after it exceeds a
> gig or two of used memory. :) I'm not sure if this is something that I
> should expect from the current gc, and when it would stop.
I don't think the GC has much to do with this; it's a lot more likely that you
have growing memory structures (in-memory logs, memo caches, etc) some-
where in your application.
adding some code that calls gc.get_objects() and dumps the N largest lists
and dictionaries might help. something like (untested):
import gc, repr, sys
def gc_check(objects):
def check(objects):
lists = []
dicts = []
for i in objects:
if isinstance(i, list): lists.append((len(i), i))
elif isinstance(i, dict): dicts.append((len(i), i))
lists.sort(); lists.reverse()
for n, i in lists[:20]:
print "LIST", n, repr.repr(i)
dicts.sort(); dicts.reverse()
for i in dicts[:20]:
print "DICT", n, repr.repr(i)
try:
check(objects)
except:
print "error in gc_check", sys.exc_info()
try:
# clear exception state (even if exc_clear doesn't exist ;-)
sys.exc_clear()
except AttributeError:
pass
(tweak as necessary)
and call this like
gc_check(gc.get_objects())
at regular intervals. if you have any constantly growing containers in your
application, they will most likely appear in the output rather quickly.
</F>
--
http://mail.python.org/mailman/listinfo/python-list
