Re: multythreading app memory consumption

2006-10-24 Thread Roman Petrichev
Thank you guys for your replies. I've just realized that there was no memory leak and it was just my mistake to think so. I've almost disappointed with my favorite programming language before addressing the problem. Actually the app consume as much memory as it should and I've just miscalculated

Re: multythreading app memory consumption

2006-10-24 Thread Andrew MacIntyre
Bryan Olson wrote: > In Python 2.5, each thread will be allocated > > thread.stack_size() > > bytes of stack address space. Note that address space is > not physical memory, nor even virtual memory. On modern > operating systems, the memory gets allocated as needed, > and 150 threads is not

Re: multythreading app memory consumption

2006-10-23 Thread Bryan Olson
Gabriel Genellina wrote: > At Sunday 22/10/2006 20:31, Roman Petrichev wrote: > >> I've just faced with very nasty memory consumption problem. >> I have a multythreaded app with 150 threads which use the only and the >> same function - through urllib2 it just gets the web page's html code >> and a

Re: multythreading app memory consumption

2006-10-23 Thread Bryan Olson
Dennis Lee Bieber wrote: > How much stack space gets allocated for 150 threads? In Python 2.5, each thread will be allocated thread.stack_size() bytes of stack address space. Note that address space is not physical memory, nor even virtual memory. On modern operating systems, the memo

Re: multythreading app memory consumption

2006-10-23 Thread Bryan Olson
Roman Petrichev wrote: > Hi folks. > I've just faced with very nasty memory consumption problem. > I have a multythreaded app with 150 threads [...] > > The test app code: > > > Q = Queue.Queue() > for i in rez: #rez length - 5000 > Q.put(i) > > > def checker(): > while True: >

Re: multythreading app memory consumption

2006-10-23 Thread Neil Hodgson
Roman Petrichev: > Dennis Lee Bieber wrote: >> How much stack space gets allocated for 150 threads? > Actually I don't know. How can I get to know this? On Linux, each thread will often be allocated 10 megabytes of stack. This can be viewed and altered with the ulimit command. Neil -

Re: multythreading app memory consumption

2006-10-23 Thread Gabriel Genellina
At Sunday 22/10/2006 20:31, Roman Petrichev wrote: I've just faced with very nasty memory consumption problem. I have a multythreaded app with 150 threads which use the only and the same function - through urllib2 it just gets the web page's html code and assigns it to local variable. On the nex

Re: multythreading app memory consumption

2006-10-23 Thread Istvan Albert
Roman Petrichev wrote: > try: > url = Q.get() > except Queue.Empty: > break This code will never raise the Queue.Empty exception. Only a non-blocking get does: url = Q.get(block=False) As mentioned before you should post working code if you expect peo

Re: multythreading app memory consumption

2006-10-23 Thread Roman Petrichev
Dennis Lee Bieber wrote: > On Mon, 23 Oct 2006 03:31:28 +0400, Roman Petrichev <[EMAIL PROTECTED]> > declaimed the following in comp.lang.python: > >> Hi folks. >> I've just faced with very nasty memory consumption problem. >> I have a multythreaded app with 150 threads which use the only and the