Re: Debugging memory leaks

2013-06-20 Thread Frank Millman
"writeson" wrote in message news:09917103-b35e-4728-8fea-bcb4ce2bd...@googlegroups.com... > Hi all, > > I've written a program using Twisted that uses SqlAlchemy to access a > database using threads.deferToThread(...) and SqlAlchemy's > scoped_session(...). This program runs for a long time, b

Re: Debugging memory leaks

2013-06-15 Thread dieter
Chris Angelico writes: > ... > Right. Everything needs to be scaled. Everything needs to be in > perspective. Losing 1 kilobit per day is indeed trivial; even losing > one kilobyte per day, which is what I assume you meant :), isn't > significant. But it's not usually per day, it's per leaking act

Re: Debugging memory leaks

2013-06-15 Thread Chris Angelico
On Sat, Jun 15, 2013 at 9:35 PM, rusi wrote: > On Jun 15, 4:23 pm, Ben Finney wrote: >> rusi writes: >> > On Jun 15, 5:16 am, Ben Finney wrote: >> > > Is a web browser a “typical desktop app”? A filesystem browser? An >> > > instant messenger? A file transfer application? A podcatcher? All of >

Re: Debugging memory leaks

2013-06-15 Thread rusi
On Jun 15, 4:23 pm, Ben Finney wrote: > rusi writes: > > On Jun 15, 5:16 am, Ben Finney wrote: > > > Is a web browser a “typical desktop app”? A filesystem browser? An > > > instant messenger? A file transfer application? A podcatcher? All of > > > those typically run for months at a time on my

Re: Debugging memory leaks

2013-06-15 Thread Ben Finney
rusi writes: > On Jun 15, 5:16 am, Ben Finney wrote: > > Is a web browser a “typical desktop app”? A filesystem browser? An > > instant messenger? A file transfer application? A podcatcher? All of > > those typically run for months at a time on my desktop. > > > > Any memory leak in any of those

Re: Debugging memory leaks

2013-06-15 Thread Chris Angelico
On Sat, Jun 15, 2013 at 4:52 PM, dieter wrote: > Chris Angelico writes: > >> ... >> It's terrible advice in generality, because it encourages a sloppiness >> of thinking: "Memory usage doesn't matter, we'll just instruct people >> to reset everything now and then". > > "Memory usage" may matter.

Re: Debugging memory leaks

2013-06-14 Thread dieter
Chris Angelico writes: > ... > It's terrible advice in generality, because it encourages a sloppiness > of thinking: "Memory usage doesn't matter, we'll just instruct people > to reset everything now and then". "Memory usage" may matter. But if you loose 1 kb a day, your process can run 3 years

Re: Debugging memory leaks

2013-06-14 Thread rusi
On Jun 15, 5:16 am, Ben Finney wrote: > rusi writes: > > On Jun 14, 1:15 am, Giorgos Tzampanakis > > wrote: > > > Am I the only one who thinks this is terrible advice? > > > I would expect a typical desktop app to run for a couple of hours -- > > maybe a couple of days. > > Is a web browser a “t

Re: Debugging memory leaks

2013-06-14 Thread Steven D'Aprano
On Fri, 14 Jun 2013 22:57:24 +, Giorgos Tzampanakis wrote: > On 2013-06-14, Steven D'Aprano wrote: > >> On Thu, 13 Jun 2013 20:15:42 +, Giorgos Tzampanakis wrote: >> Therefore: if the leak seems to be small, it may be much more advicable to restart your process periodically (dur

Re: Debugging memory leaks

2013-06-14 Thread Ben Finney
rusi writes: > On Jun 14, 1:15 am, Giorgos Tzampanakis > wrote: > > Am I the only one who thinks this is terrible advice? > > I would expect a typical desktop app to run for a couple of hours -- > maybe a couple of days. Is a web browser a “typical desktop app”? A filesystem browser? An instant

Re: Debugging memory leaks

2013-06-14 Thread Chris Angelico
On Fri, Jun 14, 2013 at 11:53 PM, rusi wrote: > On Jun 14, 1:15 am, Giorgos Tzampanakis > wrote: >> Am I the only one who thinks this is terrible advice? > > I would expect a typical desktop app to run for a couple of hours -- > maybe a couple of days. > Living with a small (enough) leak there ma

Re: Debugging memory leaks

2013-06-14 Thread Giorgos Tzampanakis
On 2013-06-14, Steven D'Aprano wrote: > On Thu, 13 Jun 2013 20:15:42 +, Giorgos Tzampanakis wrote: > >>> Therefore: if the leak seems to be small, it may be much more advicable >>> to restart your process periodically (during times where a restart does >>> not hurt much) rather than try to fin

Re: Debugging memory leaks

2013-06-14 Thread rusi
On Jun 14, 1:15 am, Giorgos Tzampanakis wrote: > Am I the only one who thinks this is terrible advice? I would expect a typical desktop app to run for a couple of hours -- maybe a couple of days. Living with a small (enough) leak there may be ok. [In particular I believe that most commercial apps

Re: Debugging memory leaks

2013-06-14 Thread rusi
On Jun 13, 6:24 am, writeson wrote: > Anyway, my real question is how to go about debugging memory leak problems in > Python, particularly for a long running > server process written with Twisted. I'm not sure how to use heapy or guppy, > and objgraph doesn't tell me enough to > locate the probl

Re: Debugging memory leaks

2013-06-14 Thread Chris Angelico
If I have not found the leak in one week, or found and fixed it in three > weeks, then I'll probably never fix it and I should just give up and > apply palliative reboots to work around the problem." > > Either that or hire an expert at debugging memory leaks. It's terrible

Re: Debugging memory leaks

2013-06-13 Thread Steven D'Aprano
ks, then I'll probably never fix it and I should just give up and apply palliative reboots to work around the problem." Either that or hire an expert at debugging memory leaks. -- Steven -- http://mail.python.org/mailman/listinfo/python-list

Re: Debugging memory leaks

2013-06-13 Thread Chris Angelico
On Fri, Jun 14, 2013 at 6:15 AM, Giorgos Tzampanakis wrote: > On 2013-06-13, dieter wrote: >> Therefore: if the leak seems to be small, it may be much more advicable >> to restart your process periodically (during times where a restart does >> not hurt much) rather than try to find (and fix) the l

Re: Debugging memory leaks

2013-06-13 Thread Steve Simmons
Giorgos Tzampanakis wrote: >On 2013-06-13, dieter wrote: > >>> ... Anyway, my real question is how to go about debugging memory >leak >>> problems in Python, particularly for a long running server process >>> written with Twisted. I'm not sure how to use heapy or guppy, and >>> objgraph doesn't

Re: Debugging memory leaks

2013-06-13 Thread Giorgos Tzampanakis
On 2013-06-13, dieter wrote: >> ... Anyway, my real question is how to go about debugging memory leak >> problems in Python, particularly for a long running server process >> written with Twisted. I'm not sure how to use heapy or guppy, and >> objgraph doesn't tell me enough to locate the problem

Re: Debugging memory leaks

2013-06-13 Thread Dave Angel
On 06/13/2013 02:07 PM, writeson wrote: Dieter, Thanks for the response, and you're correct, debugging memory leaks is tough! So far I haven't had much luck other than determining I have a leak. I've used objgraph to see that objects are being created that don't seem to ge

Re: Debugging memory leaks

2013-06-13 Thread writeson
Dieter, Thanks for the response, and you're correct, debugging memory leaks is tough! So far I haven't had much luck other than determining I have a leak. I've used objgraph to see that objects are being created that don't seem to get cleaned up. What I can't figure ou

Re: Debugging memory leaks

2013-06-12 Thread dieter
writeson writes: > ... > Anyway, my real question is how to go about debugging memory leak problems in > Python, particularly for a long running server process written with Twisted. > I'm not sure how to use heapy or guppy, and objgraph doesn't tell me enough > to locate the problem. Analysin

Debugging memory leaks

2013-06-12 Thread writeson
Hi all, I've written a program using Twisted that uses SqlAlchemy to access a database using threads.deferToThread(...) and SqlAlchemy's scoped_session(...). This program runs for a long time, but leaks memory slowly to the point of needing to be restarted. I don't know that the SqlAlchemy/thre