Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread Grant Edwards
On 2007-04-26, Martin v. Löwis <[EMAIL PROTECTED]> wrote: >> I'm a bit fuzzy on this, but I don't think there _is_ a >> practical way to "return memory to the OS" in many OSes. > > That's not true at all. Most C libraries these days manage > to return memory to the operating system. [...] Thanks

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread Magnus Lycka
[EMAIL PROTECTED] wrote: > Our (python-)macro uses massively nested loops which are unfortunately > necessary. These loops perform complex calculations in this commercial > tool. To give you a quick overview how long this macros runs: > > The outer loop takes 5-7 hours for one cycle. Each cycle cr

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread Martin v. Löwis
> The moment I close the application that launched the macro, my > ressources get freed. > > So is there a way to free my memory inside my nested loops? Yes. Most likely, it is your algorithm itself which is flawed (not Python, not the application that embeds Python): You keep, most likely, refer

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread Martin v. Löwis
> I'm a bit fuzzy on this, but I don't think there _is_ a > practical way to "return memory to the OS" in many OSes. That's not true at all. Most C libraries these days manage to return memory to the operating system. On Win32 (which the OP is most likely to use), the operating system offers the

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread TimC
"Diez B. Roggisch" <[EMAIL PROTECTED]> wrote: > Maybe delegating the actual processing to an external python > process you feed through the macro might work. Thanks for the idea. I'll check it out soon and will report about possible improvements. -- http://mail.python.org/mailman/listinfo/pytho

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-26 Thread Paul McGuire
You might try looking at references between objects, especially if there are any cyclic refs. For instance, if you have a data structure in which child nodes have back refs to their parents, try changing these to use weakref's. This may help the garbage collector to better reclaim discarded objec

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Charles Sanders
Grant Edwards wrote: > Assuming the python interpreter free()s the memory, my > understanding is that on Unixes the memory is returned to the > pool used by malloc(), but is not returned to the OS since > there isn't a practical way to ensure that the memory at the > "end" of the data segment is no

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Diez B. Roggisch
TimC schrieb: > Donald 'Paddy' McCarthy <[EMAIL PROTECTED]> wrote > >> Could you split the program into one handling the outer loop and >> calling another program, with data transfer, to handle the inner >> loops? >> >> - Paddy. > > I'm afraid this isn't possible, because the python macro is call

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread TimC
Donald 'Paddy' McCarthy <[EMAIL PROTECTED]> wrote > Could you split the program into one handling the outer loop and > calling another program, with data transfer, to handle the inner > loops? > > - Paddy. I'm afraid this isn't possible, because the python macro is called and started from within

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Donn Cave
In article <[EMAIL PROTECTED]>, Steven Howe <[EMAIL PROTECTED]> wrote: > Interesting questions. What happens when an object is 'cleaned' up by > using the 'del' command. Does the memory space stay in the python > process, get handed back to the OS, or some combination of both? > I remember 'C' on

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Grant Edwards
On 2007-04-25, Steven Howe <[EMAIL PROTECTED]> wrote: >> I'm a bit fuzzy on this, but I don't think there _is_ a >> practical way to "return memory to the OS" in many OSes. For >> example in Unix the C library uses the sbrk() call to increase >> the size of the data segment when additional memory

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Donald 'Paddy' McCarthy
[EMAIL PROTECTED] wrote: > So I read quite a few things about this phenomenon in Python 2.4.x but > I can hardly believe that there is really no solution to my problem. > > We use a commercial tool that has a macro functionality. These macros > are written in python. So far nothing extraordinary.

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Steven Howe
Grant Edwards wrote: > On 2007-04-25, Chris Mellon <[EMAIL PROTECTED]> wrote: > > >> "Returning memory to the OS" doesn't affect exhaustion of your virtual >> address space. More likely, your nested loops are just creating more >> objects than is possible to hold within a single process. >>

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Grant Edwards
On 2007-04-25, Chris Mellon <[EMAIL PROTECTED]> wrote: > "Returning memory to the OS" doesn't affect exhaustion of your virtual > address space. More likely, your nested loops are just creating more > objects than is possible to hold within a single process. I'm a bit fuzzy on this, but I don't t

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Chris Mellon
On 25 Apr 2007 15:04:59 GMT, TimC <[EMAIL PROTECTED]> wrote: > Larry Bates <[EMAIL PROTECTED]> wrote: > > > Let's see for this I need to get out my crystal ball... > > > > If it is a commercial application, you should contact their tech > > support for a solution. The problem isn't specifically a

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread TimC
Larry Bates <[EMAIL PROTECTED]> wrote: > Let's see for this I need to get out my crystal ball... > > If it is a commercial application, you should contact their tech > support for a solution. The problem isn't specifically a Python > problem but rather an implementation problem with their app. >

Re: Python not giving free memory back to the os get's me in real problems ...

2007-04-25 Thread Larry Bates
[EMAIL PROTECTED] wrote: > So I read quite a few things about this phenomenon in Python 2.4.x but > I can hardly believe that there is really no solution to my problem. > > We use a commercial tool that has a macro functionality. These macros > are written in python. So far nothing extraordinary.