On 1/17/14 10:43 AM, Jonathan Vanasco wrote:
some recent changes to a very-happy twisted daemon have resulted in a process
that grows in memory until it crashes the box. boo!
looking through the code and logs, i'm wondering if i''ve coded things in such
a way that defferds or deferrd lists are somehow not getting cleaned up if an
unhandled exception occurs.
i've been looking through all my former notes and some questions on stack
overflow, and I've seen a lot of info on using heapy and other tools to find
issues on a function-by-function basis.
i'm wondering if anyone has experience in simply monitoring the lifecycle of
deferreds ?
Hi
A few years ago I ran into the problem of non collectable cycles being
produced by keeping references around in objects locally.
Please find attached a snippet of code I used over and over again for
hunting down such cycles. It's for sure not a simple cure, but with a
manhole service being run in your process of question, it gives you the
telltale signs your looking for.
Use it like
from dumpObjects import dumpObjects
dumpObjects()
Various include/exclude sets should help you to narrow the search in
consecutive runs.
As for the lifecycle of deferreds I never ever had a problem with
deferreds not being cleaned up, it has always been me who produced those
non collectable cycles, usually under the false assumption, that it was
safe to keep a ref handy.
Werner
#! /usr/bin/env python
# -*- coding: utf-8 -*-
"""
Sucking out and dumping information out of the sea of objects
"""
import os, sys, re, gc, weakref
exc = [
"function",
"type",
"list",
"dict",
"tuple",
"wrapper_descriptor",
"module",
"method_descriptor",
"member_descriptor",
"instancemethod",
"builtin_function_or_method",
"frame",
"classmethod",
"classmethod_descriptor",
"_Environ",
"MemoryError",
"_Printer",
"_Helper",
"getset_descriptor",
"weakreaf"
]
inc = [
]
prev = {}
def dumpObjects(delta=True, limit=0, include=inc, exclude=[]):
global prev
if include != [] and exclude != []:
print 'cannot use include and exclude at the same time'
return
print 'working with:'
print ' delta: ', delta
print ' limit: ', limit
print ' include: ', include
print ' exclude: ', exclude
objects = {}
gc.collect()
oo = gc.get_objects()
for o in oo:
if getattr(o, "__class__", None):
name = o.__class__.__name__
if ((exclude == [] and include == []) or \
(exclude != [] and name not in exclude) or \
(include != [] and name in include)):
objects[name] = objects.get(name, 0) + 1
## if more:
## print o
pk = prev.keys()
pk.sort()
names = objects.keys()
names.sort()
for name in names:
if limit == 0 or objects[name] > limit:
if not prev.has_key(name):
prev[name] = objects[name]
dt = objects[name] - prev[name]
if delta or dt != 0:
print '%0.6d -- %0.6d -- ' % (dt, objects[name]), name
prev[name] = objects[name]
def getObjects(oname):
"""
gets an object list with all the named objects out of the sea of
gc'ed objects
"""
olist = []
objects = {}
gc.collect()
oo = gc.get_objects()
for o in oo:
if getattr(o, "__class__", None):
name = o.__class__.__name__
if (name == oname):
olist.append(o)
return olist
_______________________________________________
Twisted-Python mailing list
Twisted-Python@twistedmatrix.com
http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
