On Thu, 2010-07-15 at 08:06 -0400, Itamar Turner-Trauring wrote: > On Thu, 2010-07-15 at 10:46 +0100, Reza Lotun wrote: > > > As for connecting to hosts that resolve to multiple A records - I > > presume as a means of load balancing via DNS round robin
We're actually using it to provide redundancy in this instance. In our application any request for any site can be made to any (live) server, so having dead servers in the pool of A records doesn't matter so long as real web browsers failover to some other A record within a second, which they do! http://crypto.stanford.edu/dns/dns-rebinding.pdf The problem is that my test application uses client.getPage which, because it uses the reactor's standard DNS lookup mechanism, picks just one A record and sticks to it. So, it reports connection errors (some fraction of the time, as A records are randomised) even when the user of a "real" web browser would not experience them. These errors go away when the dead server(s) drop out of the DNS pool and reactor's lookups stops returning the dead IP, but this takes some time. > Gar. I should read better. Twisted uses a threadpool of gethostname by > default, but you can plug in your own resolver (e.g. you can use > twisted.names): > > http://twistedmatrix.com/documents/10.1.0/api/twisted.internet.interfaces.IReactorPluggableResolver.html#installResolver > > The question is whether the client code re-resolves on each re-connect, > and whether the current lookup interface is sufficient for this use > case. > > Alas, I'm pretty sure the answer is no. > > You could however always just do the DNS lookup yourself, passing > resulting correct IP to connectTCP, just make sure you don't block (e.g. > by using deferToThread to call gethostbyname_ex). Thanks Itamar, this is massively useful. I'll try subclassing twisted.web.client.Agent to do its own DNS lookups with twisted.names so as to be aware of the full list of A records returned. It would then attempt all the IP addresses in turn until it finds one which works, giving up only if all the IPs yield connection errors. This should mirror the behaviour of the majority of web browsers "in the wild". Would you be interested in having this code contributed back to Twisted if I can get it working? It might be a useful addition to the Agent. -- Best Regards, Luke Marsden Hybrid Logic Ltd. Web: http://www.hybrid-cluster.com/ Hybrid Web Cluster - cloud web hosting based on FreeBSD and ZFS Mobile: +447791750420 _______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
