Przemysław Różycki wrote:
Hello,
I have written some code, which creates many threads for each connection
('main connection'). The purpose of this code is to balance the load
between several connections ('pipes'). The number of spawned threads
depends on how many pipes I create (= 2*n+2, where n is the number of
pipes).
For good results I'll presumably share main connection's load between 10
pipes - therefore 22 threads will be spawned. Now if about 50
connections are forwarded the number of threads rises to thousand of
threads (or several thousands if even more connections are established).
My questions are:
- What is the cost (in memory / CPU usage) of creating such amounts of
threads?
- Is there any 'upper boundary' that limits the number of threads? (is
it python / OS related)
- Is that the sign of 'clumsy programming' - i.e. Is creating so many
threads a bad habit? (I must say that it simplified the solution of my
problem very much).
Limiting the number of threads is possible, but would affect the
independence of data flows. (ok I admit - creating tricky algorithm
could perhaps gurantee concurrency without spawning so many threads -
but it's the simplest solution to this problem :) ).
PR,
I notice there's a resource module with a
getrusage(who) that looks like it would support
a test to get what you need.
wes
--
http://mail.python.org/mailman/listinfo/python-list
