On Oct 6, 2009, at 11:25 PM, Andrew Bennetts wrote: > Steve Steiner (listsin) wrote: > [...] >> I expect the usual flurry of "you must post your exact code or we >> can't help you at all, moron" posts, but... > > I'll try to restrain myself ;)
Thanks, I appreciate your restraint. Must say, most posts without code drive me a little cuckoo, too. I hope this one was justified in not including specific code. >> My two top guesses are: > > 1) the web server failing to cope with thousands of concurrent > requests > gracefully, or Ah, that would certainly make sense in this particular case. I am asking the server for an *awful* lot of stuff all at one time. > 2) the number of sockets is hitting a system limit (number of FDs > you can pass > to select(), or hitting the max number of file descriptors, > something like > that) That may also be an issue, thank you for pointing that out. > IIRC the RFCs say that clients SHOULD use no more than two > concurrent connections to a server... As you said, the performance is going to be rate-limited by the server's ability to respond to requests anyway, so I think what I'll do is just make it so that the "get me a page" queue doesn't put more than a couple of requests "in-flight" at the same time. Thanks for helping me think this out! Thanks, S _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
