> > MRAB wrote: > > Erm, shurely the bottleneck will be bandwidth not processor/memory?* If it > isn't then - yes, you run the risk of actually DOSing their servers!
> > Your mac will run thousands of threads comfortably but your router may not > handle the thousands of TCP/IP connections you throw at it very well, > especially if it is a domestic model, and sure as hell sourceforge aren't > going to want more than a handfull of concurrent connections from you. > > Typical sourceforge page ~ 30K > Project pages to read = 240000 > > = ~6.8 Gigabytes The best path for avoid congestion network problem and probably "DOS attack" it's use a equilibrate number of parallel connection with a number of threads. One router can handle hundred connections and Mac thread implementation may be also - with a lot of memory or stack size profiling. If you run hundred threads you will be fight with OS, Webserver, Router and network starvation problems. I believe use a 64 number of threads it's ok, but probably all 64 thread connections will be performance for the same sourceforge web server - balancing algorithm - and may be that this week up suspicion to the admin. May be, you can use httplib [1] for do a persistent connection and reuse same connection for same thread. [1] http://www.python.org/doc/2.4.2/lib/httplib-examples.html -- Pau Freixes Linux GNU/User
-- http://mail.python.org/mailman/listinfo/python-list
