Re: ulimit stack size and python threads

2009-01-09 Thread Martin v. Löwis
> I see. I should be blaming the default behavior of pthreads. You shouldn't blame anybody. Instead, you should sit down and study the problem in detail, until you fully understand it. Then you should start contributing fixes. Never ever should you spread blame. Regards, Martin -- http://mail.py

Re: ulimit stack size and python threads

2009-01-09 Thread Martin v. Löwis
> Always crashing because I asked the OS to please not allow a process > to grow too big is what I call overloading the meaning of ulimit -s. Please trust that there is no explicit code in the Python interpreter that tests whether the stack size is 4GB, and then produces an explicit crash. > It's

Re: ulimit stack size and python threads

2009-01-08 Thread Greg Lindahl
I see. I should be blaming the default behavior of pthreads. I did work on a OpenMP library once, and we worked around this problem, plus we gave good error messages. Given the number of HPC sites which use Python, I'd think that Python would have grown similar features. (HPC sites are more likely

Re: ulimit stack size and python threads

2009-01-08 Thread Andrew MacIntyre
Greg Lindahl wrote: I figure this is a FAQ, but I can't find it in any FAQs. I want to limit the stacksize on my server. If I set it to 8 megs, or unlimited, python is happy. If I set it to 4 gigabytes, things like yum (which is a python program) crash creating a thread. This is on an x86_64 l

Re: ulimit stack size and python threads

2009-01-08 Thread Greg Lindahl
> Why do you think Python is overloading the meaning of that? I ensure > you it isn't - it doesn't actively care what the limits are. Always crashing because I asked the OS to please not allow a process to grow too big is what I call overloading the meaning of ulimit -s. It's quite surprising. Not

Re: ulimit stack size and python threads

2009-01-08 Thread Martin v. Löwis
> But even if that worked, I'd be worried that python is doing something > bad with the ulimit -s value under the covers. Again: it definitely isn't. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list

Re: ulimit stack size and python threads

2009-01-08 Thread Greg Lindahl
> How much higher? You could try just under 4GB (unsigned 32-bit) and just > under 2GB (signed 32-bit). I'd like to set it to be about 1/2 the memory size of my server, which happens to end up being 4 gbytes. And no, slightly less than 4 gb doesn't work. But even if that worked, I'd be worried th

Re: ulimit stack size and python threads

2009-01-08 Thread Martin v. Löwis
> Why is Python overloading the meaning of the ulimit -s like this? Why do you think Python is overloading the meaning of that? I ensure you it isn't - it doesn't actively care what the limits are. Regards, Martin -- http://mail.python.org/mailman/listinfo/python-list

Re: ulimit stack size and python threads

2009-01-08 Thread MRAB
Greg Lindahl wrote: I'm only guessing, but could it be a 32-bit limit somewhere? Have you tried, say, 1GB, which would be within a 32-bit limit? Indeed, ulimit -s 100 (a bit smaller than 1 GB) does work, but it doesn't solve my problem, since I want to set the limit higher than 1 GB. How m

Re: ulimit stack size and python threads

2009-01-08 Thread Greg Lindahl
> I'm only guessing, but could it be a 32-bit limit somewhere? Have you > tried, say, 1GB, which would be within a 32-bit limit? Indeed, ulimit -s 100 (a bit smaller than 1 GB) does work, but it doesn't solve my problem, since I want to set the limit higher than 1 GB. -- greg -- http://mail.p

Re: ulimit stack size and python threads

2009-01-08 Thread MRAB
Greg Lindahl wrote: I figure this is a FAQ, but I can't find it in any FAQs. I want to limit the stacksize on my server. If I set it to 8 megs, or unlimited, python is happy. If I set it to 4 gigabytes, things like yum (which is a python program) crash creating a thread. This is on an x86_64 l