In article <[EMAIL PROTECTED]>,
=?ISO-8859-2?Q?Przemys=B3aw_R=F3=BFycki?=
<[EMAIL PROTECTED]> wrote:
>
>I have written some code, which creates many threads for each connection
>('main connection'). The purpose of this code is to balance the load
>between several connections ('pipes'). The number of spawned threads
>depends on how many pipes I create (= 2*n+2, where n is the number of
>pipes).
>
>For good results I'll presumably share main connection's load between 10
>pipes - therefore 22 threads will be spawned. Now if about 50
>connections are forwarded the number of threads rises to thousand of
>threads (or several thousands if even more connections are established).
I'm a bit confused by your math. Fifty connections should be 102
threads, which is quite reasonable.
>My questions are:
>- What is the cost (in memory / CPU usage) of creating such amounts of
>threads?
>- Is there any 'upper boundary' that limits the number of threads? (is
>it python / OS related)
>- Is that the sign of 'clumsy programming' - i.e. Is creating so many
>threads a bad habit? (I must say that it simplified the solution of my
>problem very much).
>
>Limiting the number of threads is possible, but would affect the
>independence of data flows. (ok I admit - creating tricky algorithm
>could perhaps gurantee concurrency without spawning so many threads -
>but it's the simplest solution to this problem :) ).
My experience with lots of threads dates back to Python 1.5.2, but I
rarely saw much improvement with more than a hundred threads, even for
heavily I/O-bound applications on a multi-CPU system. However, if your
focus is algorithmic complexity, you should be able to handle a couple of
thousand threads easily enough.
--
Aahz ([EMAIL PROTECTED]) <*> http://www.pythoncraft.com/
"The joy of coding Python should be in seeing short, concise, readable
classes that express a lot of action in a small amount of clear code --
not in reams of trivial code that bores the reader to death." --GvR
--
http://mail.python.org/mailman/listinfo/python-list
