Re: script uses up all memory

2014-03-14 Thread Kent Engström
Larry Martell writes: > I figured out what is causing this. Each pass through the loop it does: > > self.tools = Tool.objects.filter(ip__isnull=False) > > And that is what is causing the memory consumption. If I move that > outside the loop and just do that once the memory issue goes away. Now > I

Re: script uses up all memory

2014-03-06 Thread Marko Rauhamaa
Chris Angelico : > On Fri, Mar 7, 2014 at 10:53 AM, Marko Rauhamaa wrote: >>class MyStateMachine: >>def __init__(self): >>sm = self >> >>class IDLE: >>def ding(self): >>sm.open_door() >>sm.state = AT_DOOR(

Re: script uses up all memory

2014-03-06 Thread Chris Angelico
On Fri, Mar 7, 2014 at 10:53 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> Can you give a useful example of a closure that does create a refloop? > > Just the other day, I mentioned the state pattern: > >class MyStateMachine: >def __init__(self): >sm = self > >

Re: script uses up all memory

2014-03-06 Thread Marko Rauhamaa
Chris Angelico : > Can you give a useful example of a closure that does create a refloop? Just the other day, I mentioned the state pattern: class MyStateMachine: def __init__(self): sm = self class IDLE: def ding(self): sm.open_

Re: script uses up all memory

2014-03-06 Thread Chris Angelico
On Fri, Mar 7, 2014 at 10:12 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> I think this thread is proof that they are to be avoided. The GC >> wasn't doing its job unless explicitly called on. The true solution is >> to break the refloop; the quick fix is to call gc.collect(). I stand >> by th

Re: script uses up all memory

2014-03-06 Thread Marko Rauhamaa
Chris Angelico : > I think this thread is proof that they are to be avoided. The GC > wasn't doing its job unless explicitly called on. The true solution is > to break the refloop; the quick fix is to call gc.collect(). I stand > by the recommendation to put an explanatory comment against the > co

Re: script uses up all memory

2014-03-06 Thread Chris Angelico
On Fri, Mar 7, 2014 at 9:34 AM, Marko Rauhamaa wrote: > Chris Angelico : > >> Not all problems need to be solved perfectly :) But at very least, I >> would put a comment against your collect() call explaining what >> happens: that self.tools is involved in a refloop. Most Python code >> shouldn't

Re: script uses up all memory

2014-03-06 Thread Chris Angelico
On Fri, Mar 7, 2014 at 9:21 AM, Larry Martell wrote: > First I added del(self.tools) before the Django call. That did not > stop the memory consumption. Then I added a call to gc.collect() after > the del and that did solve it. gc.collect() returns 0 each time, so > I'm going to declare victory an

Re: script uses up all memory

2014-03-06 Thread Marko Rauhamaa
Chris Angelico : > Not all problems need to be solved perfectly :) But at very least, I > would put a comment against your collect() call explaining what > happens: that self.tools is involved in a refloop. Most Python code > shouldn't have to call gc.collect(), so it's worth explaining why you >

Re: script uses up all memory

2014-03-06 Thread Larry Martell
On Thu, Mar 6, 2014 at 5:11 PM, Chris Angelico wrote: > On Fri, Mar 7, 2014 at 8:56 AM, Larry Martell wrote: >> I figured out what is causing this. Each pass through the loop it does: >> >> self.tools = Tool.objects.filter(ip__isnull=False) >> >> And that is what is causing the memory consumption

Re: script uses up all memory

2014-03-06 Thread Chris Angelico
On Fri, Mar 7, 2014 at 9:07 AM, Larry Martell wrote: > Apparently the object returned by that call is immutable as if I look > at id(self.tools) each pass through the loop, it is different. Is > there some way I can recover that memory? Not sure what mutability has to do with that. The changing i

Re: script uses up all memory

2014-03-06 Thread Chris Angelico
On Fri, Mar 7, 2014 at 8:56 AM, Larry Martell wrote: > I figured out what is causing this. Each pass through the loop it does: > > self.tools = Tool.objects.filter(ip__isnull=False) > > And that is what is causing the memory consumption. If I move that > outside the loop and just do that once the

Re: script uses up all memory

2014-03-06 Thread Larry Martell
On Thu, Mar 6, 2014 at 4:56 PM, Larry Martell wrote: > On Wed, Mar 5, 2014 at 5:27 PM, Larry Martell wrote: >> I have a script that forks off other processes and attempts to manage >> them. Here is a stripped down version of the script: >> >> self.sleepTime = 300 >> self.procs = {

Re: script uses up all memory

2014-03-06 Thread Larry Martell
On Wed, Mar 5, 2014 at 5:27 PM, Larry Martell wrote: > I have a script that forks off other processes and attempts to manage > them. Here is a stripped down version of the script: > > self.sleepTime = 300 > self.procs = {} > self.startTimes = {} > self.cmd = ['pytho

Re: script uses up all memory

2014-03-05 Thread Larry Martell
On Wed, Mar 5, 2014 at 7:33 PM, Chris Angelico wrote: > On Thu, Mar 6, 2014 at 11:20 AM, Larry Martell > wrote: >> On Wed, Mar 5, 2014 at 5:39 PM, Chris Angelico wrote: >>> On Thu, Mar 6, 2014 at 9:27 AM, Larry Martell >>> wrote: I have a script that forks off other processes and attempt

Re: script uses up all memory

2014-03-05 Thread Chris Angelico
On Thu, Mar 6, 2014 at 11:20 AM, Larry Martell wrote: > On Wed, Mar 5, 2014 at 5:39 PM, Chris Angelico wrote: >> On Thu, Mar 6, 2014 at 9:27 AM, Larry Martell >> wrote: >>> I have a script that forks off other processes and attempts to manage >>> them. Here is a stripped down version of the scr

Re: script uses up all memory

2014-03-05 Thread Larry Martell
On Wed, Mar 5, 2014 at 5:39 PM, Chris Angelico wrote: > On Thu, Mar 6, 2014 at 9:27 AM, Larry Martell wrote: >> I have a script that forks off other processes and attempts to manage >> them. Here is a stripped down version of the script: >> >> self.sleepTime = 300 > > That's not a stand-a

Re: script uses up all memory

2014-03-05 Thread Chris Angelico
On Thu, Mar 6, 2014 at 9:27 AM, Larry Martell wrote: > I have a script that forks off other processes and attempts to manage > them. Here is a stripped down version of the script: > > self.sleepTime = 300 That's not a stand-alone script. What environment is it running in? Can you reproduc

script uses up all memory

2014-03-05 Thread Larry Martell
I have a script that forks off other processes and attempts to manage them. Here is a stripped down version of the script: self.sleepTime = 300 self.procs = {} self.startTimes = {} self.cmd = ['python', '/usr/local/motor/motor/app/some_other_script.py'] whi