Am 28.07.2011 23:07, schrieb smith jack: > I start many threads in order to make the work done, when the > concurrent number is set to 300, all thing just works fine, but when > the number is set to 350 or higher, error just comes out? what's wrong > ? the error info is just as follows: failed to start . > > I am confused, does this have something to do with the operating > system, i am now using Linux, any suggestion to make the system to > support more python threads?
300 threads is an insanely high amount of threads for almost every environment. For CPython threads won't speed up your program if it's CPU bound and spends most of its time in Python space (no C code that releases the GIL). For IO bound code you are going to get much better performance with some kind of async event loop (select, poll, epoll). Twisted's event reactor may help here, too. With that many threads any program (not limited to Python) will spend a considerable amount of CPU resources to manage the threads. Every thread also consumes a fair amount of memory since every thread needs its own C stack. On most systems the C stack for every thread is about 8 MB. There are also some management things that need memory. You can reduce the stack size *before* you start any threads: import sys import thread import resource sys.setrecursionlimit(500) # default is 1000 stacksize = 4 * 1024 * 1024 # default is 8 MB on most systems thread.stack_size(stacksize) resource.setrlimit(resource.RLIMIT_STACK, (stacksize, -1)) That should give you some more spare memory. Christian -- http://mail.python.org/mailman/listinfo/python-list
