Re: [python-uk] urllib latency

2010-12-21 Thread John J Lee
On Tue, 21 Dec 2010, Matt Hamilton wrote: [...] And then re-running the tests from the same python prompt that was open just before I wrote that email and I now have the 100ms delay back again... yet not still in ab. Quitting and starting a new python process and I still have the delay. I gue

Re: [python-uk] urllib latency

2010-12-21 Thread Jon Ribbens
On Tue, Dec 21, 2010 at 10:39:23AM +, Matt Hamilton wrote: > And then re-running the tests from the same python prompt that was > open just before I wrote that email and I now have the 100ms delay > back again... yet not still in ab. Quitting and starting a new > python process and I still have

Re: [python-uk] urllib latency

2010-12-21 Thread Matt Hamilton
On 21 Dec 2010, at 10:29, Matt Hamilton wrote: > I then have a similar issue for the request I mentioned in this thread, that > happens to be cached by varnish. If I then disable delayed acks on the OSX > box running the python tests, that speeds up again and I lose the 100ms extra > latency a

Re: [python-uk] urllib latency

2010-12-21 Thread Matt Hamilton
On 20 Dec 2010, at 16:26, Doug Winter wrote: > Matt Hamilton wrote: >> I'm off to go dig in the urllib code and see if I can see anything >> there. I'm wondering if urllib is taking some time to process the >> data after it receives it before doing anything. >> >> This is on OSX, but I'm going t

Re: [python-uk] urllib latency

2010-12-20 Thread Doug Winter
On 20/12/10 17:19, Jon Ribbens wrote: Could it be SO_REUSEADDR? Um... no ;-) It did seem unlikely ;) -- Telephone: +44 1904 567330, Mobile: +44 7879 423002 Switchboard: +44 1904 567349, Fax: +44 20 79006980 Post: Tower House, Fishergate, York, YO10 4UA, UK Registered in England. Company No

Re: [python-uk] urllib latency

2010-12-20 Thread Jon Ribbens
On Mon, Dec 20, 2010 at 04:26:09PM +, Doug Winter wrote: > Matt Hamilton wrote: > > I'm off to go dig in the urllib code and see if I can see anything > > there. I'm wondering if urllib is taking some time to process the > > data after it receives it before doing anything. > > > > This is on O

Re: [python-uk] urllib latency

2010-12-20 Thread Doug Winter
Matt Hamilton wrote: > I'm off to go dig in the urllib code and see if I can see anything > there. I'm wondering if urllib is taking some time to process the > data after it receives it before doing anything. > > This is on OSX, but I'm going to go try it on a FreeBSD box as I can > then use ktrac

Re: [python-uk] urllib latency

2010-12-20 Thread Matt Hamilton
On 20 Dec 2010, at 14:15, Alex Willmer wrote: > On 20 December 2010 13:54, Matt Hamilton wrote: > Anyone know why urllib.urlopen() can be so much slower than using ab to do > the same thing? I seem to be getting an extra 100ms latency on a simple HTTP > GET request of a static, small image. >

Re: [python-uk] urllib latency

2010-12-20 Thread Matt Hamilton
On 20 Dec 2010, at 13:54, Matt Hamilton wrote: > Ok, here's a non job related post... > > Anyone know why urllib.urlopen() can be so much slower than using ab to do > the same thing? I seem to be getting an extra 100ms latency on a simple HTTP > GET request of a static, small image. Some mor

Re: [python-uk] urllib latency

2010-12-20 Thread Alex Willmer
On 20 December 2010 13:54, Matt Hamilton wrote: > Anyone know why urllib.urlopen() can be so much slower than using ab to do > the same thing? I seem to be getting an extra 100ms latency on a simple HTTP > GET request of a static, small image. > Just some possibles: - How many DNS lookups is ea

Re: [python-uk] urllib latency

2010-12-20 Thread Matt Hamilton
On 20 Dec 2010, at 14:04, Giorgio Zoppi wrote: > 2010/12/20 Matt Hamilton : >> Ok, here's a non job related post... >> >> Anyone know why urllib.urlopen() can be so much slower than using ab to do >> the same thing? I seem to be getting an extra 100ms latency on a simple HTTP >> GET request of

Re: [python-uk] urllib latency

2010-12-20 Thread Giorgio Zoppi
2010/12/20 Matt Hamilton : > Ok, here's a non job related post... > > Anyone know why urllib.urlopen() can be so much slower than using ab to do > the same thing? I seem to be getting an extra 100ms latency on a simple HTTP > GET request of a static, small image. > I've tried it repeatedly and ge

[python-uk] urllib latency

2010-12-20 Thread Matt Hamilton
Ok, here's a non job related post... Anyone know why urllib.urlopen() can be so much slower than using ab to do the same thing? I seem to be getting an extra 100ms latency on a simple HTTP GET request of a static, small image. e.g. >>> for x in range(10): ... t1 = time(); data = urlopen('htt