Re: Python memory usage

2008-10-29 Thread bieffe62
On 21 Ott, 17:19, Rolf Wester <[EMAIL PROTECTED]> wrote: > Hi, > > I have the problem that with long running Python scripts (many loops) > memory consumption increases until the script crashes. I used the > following small script to understand what might happen: > > import gc > > print len(gc.get_o

Re: Python memory usage

2008-10-29 Thread David Cournapeau
On Wed, Oct 29, 2008 at 6:56 PM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > On Oct 21, 5:19 pm, Rolf Wester <[EMAIL PROTECTED]> wrote: >> Hi, >> >> I have the problem that with long running Python scripts (many loops) >> memory consumption increases until the script crashes. I used the >> follo

Re: Python memory usage

2008-10-29 Thread [EMAIL PROTECTED]
On Oct 21, 5:19 pm, Rolf Wester <[EMAIL PROTECTED]> wrote: > Hi, > > I have the problem that with long running Python scripts (many loops) > memory consumption increases until the script crashes. I used the > following small script to understand what might happen: > AFAIK, python uses malloc behi

Re: Python Memory Usage

2007-06-30 Thread malkarouri
On Jun 20, 4:48 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > I am using Python to process particle data from a physics simulation. > There are about 15 MB of data associated with each simulation, but > there are many simulations. I read the data from each simulation into > Numpy arrays and

Re: Python memory usage

2006-11-13 Thread Jonathan Ballet
Le Mon, 13 Nov 2006 21:30:35 +0100, Fredrik Lundh <[EMAIL PROTECTED]> a écrit : > Jonathan Ballet wrote: > > >> http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object > > > > Is it still true with Python 2.5 ? > > > > I mean, [http://evanjones.ca/python-memory

Re: Python memory usage

2006-11-13 Thread Fredrik Lundh
Klaas wrote: > I think floats use obmalloc so I'm slightly surprised you don't see > differences. as noted in the FAQ I just posted a link to, floats also use a free list (using pretty much identical code to that used for integers). see comments in Objects/intobject.c (quoted below) and Object

Re: Python memory usage

2006-11-13 Thread Klaas
velotron wrote: > On Nov 9, 8:38 pm, "Klaas" <[EMAIL PROTECTED]> wrote: > > > I was referring specifically to abominations like range(100) > > However, there are plenty of valid reasons to allocate huge lists of > integers. I'm sure there are some; I doubt there are plenty. Care to name a few?

Re: Python memory usage

2006-11-13 Thread Fredrik Lundh
Jonathan Ballet wrote: >> http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object > > Is it still true with Python 2.5 ? > > I mean, [http://evanjones.ca/python-memory.html] should fix this > behaviour, doesn't it ? not really -- that change just means that Pyt

Re: Python memory usage

2006-11-13 Thread Jonathan Ballet
Le Mon, 13 Nov 2006 20:46:58 +0100, Fredrik Lundh <[EMAIL PROTECTED]> a écrit : > > http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object > > > Is it still true with Python 2.5 ? I mean, [http://evanjones.ca/python-memory.html] should fix this behaviour, do

Re: Python memory usage

2006-11-13 Thread Fredrik Lundh
velotron wrote: > x=range(1000) > x=None > > The problem exists for floats too, so for a less contrived example: > > x=[random.weibullvariate(7.0,2.0) for i in xrange(1000)] > x=None > > Both leave the Python process bloated in my environment. Is this > problem a good candidate for th

Re: Python memory usage

2006-11-13 Thread velotron
(hello group) On Nov 9, 8:38 pm, "Klaas" <[EMAIL PROTECTED]> wrote: > I was referring specifically to abominations like range(100) However, there are plenty of valid reasons to allocate huge lists of integers. This issue has been worked on: http://evanjones.ca/python-memory.html http://eva

Re: Python memory usage

2006-11-09 Thread Klaas
placid wrote: > Actually i am executing that code snippet and creating BeautifulSoup > objects in the range() (now xrange() ) code block. Right; I was referring specifically to abominations like range(100), not looping over an incrementing integer. -Mike -- http://mail.python.org/mailman

Re: Python memory usage

2006-11-07 Thread placid
Klaas wrote: > placid wrote: > > Hi All, > > > > Just wondering when i run the following code; > > > > for i in range(100): > > print i > > > > the memory usage of Python spikes and when the range(..) block finishes > > execution the memory usage does not drop down. Is there a way of > >

Re: Python memory usage

2006-11-07 Thread Klaas
placid wrote: > Hi All, > > Just wondering when i run the following code; > > for i in range(100): > print i > > the memory usage of Python spikes and when the range(..) block finishes > execution the memory usage does not drop down. Is there a way of > freeing this memory that range(..) a

Re: Python memory usage

2006-11-07 Thread placid
William Heymann wrote: > On Tuesday 07 November 2006 22:42, placid wrote: > > Hi All, > > > > Just wondering when i run the following code; > > > > for i in range(100): > > print i > > > > the memory usage of Python spikes and when the range(..) block finishes > > execution the memory usa

Re: Python memory usage

2006-11-07 Thread William Heymann
On Tuesday 07 November 2006 22:42, placid wrote: > Hi All, > > Just wondering when i run the following code; > > for i in range(100): > print i > > the memory usage of Python spikes and when the range(..) block finishes > execution the memory usage does not drop down. Is there a way of > f

Re: Python memory usage

2006-11-07 Thread Jorge Vargas
On 7 Nov 2006 21:42:31 -0800, placid <[EMAIL PROTECTED]> wrote: > Hi All, > > Just wondering when i run the following code; > > for i in range(100): > print i > the problem of that is that all the memory is used by the list returned by range which wont be freed until the for loop exits tr