Re: Python memory usage

2008-10-29 Thread bieffe62
On 21 Ott, 17:19, Rolf Wester <[EMAIL PROTECTED]> wrote: > Hi, > > I have the problem that with long running Python scripts (many loops) > memory consumption increases until the script crashes. I used the > following small script to understand what might happen: > > import gc > > print len(gc.get_o

Re: Python memory usage

2008-10-29 Thread David Cournapeau
On Wed, Oct 29, 2008 at 6:56 PM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > On Oct 21, 5:19 pm, Rolf Wester <[EMAIL PROTECTED]> wrote: >> Hi, >> >> I have the problem that with long running Python scripts (many loops) >> memory consumption increases until the script crashes. I used the >> follo

Re: Python memory usage

2008-10-29 Thread [EMAIL PROTECTED]
On Oct 21, 5:19 pm, Rolf Wester <[EMAIL PROTECTED]> wrote: > Hi, > > I have the problem that with long running Python scripts (many loops) > memory consumption increases until the script crashes. I used the > following small script to understand what might happen: > AFAIK, python uses malloc behi

Re: Python Memory Usage

2007-06-30 Thread malkarouri
On Jun 20, 4:48 am, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: > I am using Python to process particle data from a physics simulation. > There are about 15 MB of data associated with each simulation, but > there are many simulations. I read the data from each simulation into > Numpy arrays and

Python Memory Usage

2007-06-19 Thread [EMAIL PROTECTED]
I am using Python to process particle data from a physics simulation. There are about 15 MB of data associated with each simulation, but there are many simulations. I read the data from each simulation into Numpy arrays and do a simple calculation on them that involves a few eigenvalues of small m

Re: Python memory usage

2006-11-13 Thread Jonathan Ballet
Le Mon, 13 Nov 2006 21:30:35 +0100, Fredrik Lundh <[EMAIL PROTECTED]> a écrit : > Jonathan Ballet wrote: > > >> http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object > > > > Is it still true with Python 2.5 ? > > > > I mean, [http://evanjones.ca/python-memory

Re: Python memory usage

2006-11-13 Thread Fredrik Lundh
Klaas wrote: > I think floats use obmalloc so I'm slightly surprised you don't see > differences. as noted in the FAQ I just posted a link to, floats also use a free list (using pretty much identical code to that used for integers). see comments in Objects/intobject.c (quoted below) and Object

Re: Python memory usage

2006-11-13 Thread Klaas
velotron wrote: > On Nov 9, 8:38 pm, "Klaas" <[EMAIL PROTECTED]> wrote: > > > I was referring specifically to abominations like range(100) > > However, there are plenty of valid reasons to allocate huge lists of > integers. I'm sure there are some; I doubt there are plenty. Care to name a few?

Re: Python memory usage

2006-11-13 Thread Fredrik Lundh
Jonathan Ballet wrote: >> http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object > > Is it still true with Python 2.5 ? > > I mean, [http://evanjones.ca/python-memory.html] should fix this > behaviour, doesn't it ? not really -- that change just means that Pyt

Re: Python memory usage

2006-11-13 Thread Jonathan Ballet
Le Mon, 13 Nov 2006 20:46:58 +0100, Fredrik Lundh <[EMAIL PROTECTED]> a écrit : > > http://effbot.org/pyfaq/why-doesnt-python-release-the-memory-when-i-delete-a-large-object > > > Is it still true with Python 2.5 ? I mean, [http://evanjones.ca/python-memory.html] should fix this behaviour, do

Re: Python memory usage

2006-11-13 Thread Fredrik Lundh
velotron wrote: > x=range(1000) > x=None > > The problem exists for floats too, so for a less contrived example: > > x=[random.weibullvariate(7.0,2.0) for i in xrange(1000)] > x=None > > Both leave the Python process bloated in my environment. Is this > problem a good candidate for th

Re: Python memory usage

2006-11-13 Thread velotron
(hello group) On Nov 9, 8:38 pm, "Klaas" <[EMAIL PROTECTED]> wrote: > I was referring specifically to abominations like range(100) However, there are plenty of valid reasons to allocate huge lists of integers. This issue has been worked on: http://evanjones.ca/python-memory.html http://eva

Re: Python memory usage

2006-11-09 Thread Klaas
placid wrote: > Actually i am executing that code snippet and creating BeautifulSoup > objects in the range() (now xrange() ) code block. Right; I was referring specifically to abominations like range(100), not looping over an incrementing integer. -Mike -- http://mail.python.org/mailman

Re: Python memory usage

2006-11-07 Thread placid
Klaas wrote: > placid wrote: > > Hi All, > > > > Just wondering when i run the following code; > > > > for i in range(100): > > print i > > > > the memory usage of Python spikes and when the range(..) block finishes > > execution the memory usage does not drop down. Is there a way of > >

Re: Python memory usage

2006-11-07 Thread Klaas
placid wrote: > Hi All, > > Just wondering when i run the following code; > > for i in range(100): > print i > > the memory usage of Python spikes and when the range(..) block finishes > execution the memory usage does not drop down. Is there a way of > freeing this memory that range(..) a

Re: Python memory usage

2006-11-07 Thread placid
William Heymann wrote: > On Tuesday 07 November 2006 22:42, placid wrote: > > Hi All, > > > > Just wondering when i run the following code; > > > > for i in range(100): > > print i > > > > the memory usage of Python spikes and when the range(..) block finishes > > execution the memory usa

Re: Python memory usage

2006-11-07 Thread William Heymann
On Tuesday 07 November 2006 22:42, placid wrote: > Hi All, > > Just wondering when i run the following code; > > for i in range(100): > print i > > the memory usage of Python spikes and when the range(..) block finishes > execution the memory usage does not drop down. Is there a way of > f

Re: Python memory usage

2006-11-07 Thread Jorge Vargas
On 7 Nov 2006 21:42:31 -0800, placid <[EMAIL PROTECTED]> wrote: > Hi All, > > Just wondering when i run the following code; > > for i in range(100): > print i > the problem of that is that all the memory is used by the list returned by range which wont be freed until the for loop exits tr

Python memory usage

2006-11-07 Thread placid
Hi All, Just wondering when i run the following code; for i in range(100): print i the memory usage of Python spikes and when the range(..) block finishes execution the memory usage does not drop down. Is there a way of freeing this memory that range(..) allocated? I found this documen