Re: Urllib2: Only a partial page retrieved

2010-05-23 Thread Dragon Lord
Thanks, that works perfectly! (oh and I learnt something new too, because I tried using telnet to connect to the server :) ) On May 23, 11:42 am, hpsMouse wrote: > I know what the problem is. > > Server checks client's locale setting to determine how the date should > be displayed. Python don't

Re: Urllib2: Only a partial page retrieved

2010-05-23 Thread hpsMouse
I know what the problem is. Server checks client's locale setting to determine how the date should be displayed. Python don't send locale information by default. So server fails at that point. If you add the following field in the HTTP request, the response will be correct: Accept-Language: en --

Re: Urllib2: Only a partial page retrieved

2010-05-23 Thread hpsMouse
On 5月22日, 下午5时43分, Dragon Lord wrote: > The cutoff is allways at the same location: just after the label > "Meeting date" and before the date itself. Could it be that something > is interpreted as and eof command or something like that? > > example of the cutoff point with a bad page: > Meeting Da

Re: Urllib2: Only a partial page retrieved

2010-05-22 Thread Dragon Lord
Oops, het "Good" page is alos handled wrongly. The papers from 2000 are handled wrong too so a real example of a well performing page: http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=5206867 On May 22, 11:43 am, Dragon Lord wrote: > I am trying to download a few IEEE pages by using urlli