Re: Problem when fetching page using urllib2.urlopen

2009-08-11 Thread dorzey
On 10 Aug, 18:11, "Diez B. Roggisch" wrote: > dorzey wrote: > > "geturl - this returns the real URL of the page fetched. This is > > useful because urlopen (or the opener object used) may have followed a > > redirect. The URL of the page fetched may not be the same as the URL > > requested." from

Re: Problem when fetching page using urllib2.urlopen

2009-08-11 Thread Diez B. Roggisch
dorzey wrote: > "geturl - this returns the real URL of the page fetched. This is > useful because urlopen (or the opener object used) may have followed a > redirect. The URL of the page fetched may not be the same as the URL > requested." from > http://www.voidspace.org.uk/python/articles/urllib2.

Re: Problem when fetching page using urllib2.urlopen

2009-08-10 Thread jitu
Yes Piet you were right this works. But seems does not work on google app engine, since it appends it own agent info as seen below 'User-Agent': 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.13) Gecko/2009073021 Firefox/3.0.13 AppEngine-Google; (+http://code.google.com/appengi

Re: Problem when fetching page using urllib2.urlopen

2009-08-10 Thread Piet van Oostrum
> jitu (j) wrote: >j> Hi, >j> A html page contains 'anchor' elements with 'href' attribute having >j> a semicolon in the url , while fetching the page using >j> urllib2.urlopen, all such href's containing 'semicolons' are >j> truncated. >j> For example the href >http://travel.yahoo.co

Re: Problem when fetching page using urllib2.urlopen

2009-08-10 Thread dorzey
"geturl - this returns the real URL of the page fetched. This is useful because urlopen (or the opener object used) may have followed a redirect. The URL of the page fetched may not be the same as the URL requested." from http://www.voidspace.org.uk/python/articles/urllib2.shtml#info-and-geturl I

Re: Problem when fetching page using urllib2.urlopen

2009-08-10 Thread jitu
On Aug 10, 4:39 pm, jitu wrote: > Hi, > > A html page  contains 'anchor' elements with 'href' attribute  having > a semicolon  in the url , while fetching the page using > urllib2.urlopen, all such href's  containing  'semicolons' are > truncated. > > For example the > hrefhttp://travel.yahoo.com

Problem when fetching page using urllib2.urlopen

2009-08-10 Thread jitu
Hi, A html page contains 'anchor' elements with 'href' attribute having a semicolon in the url , while fetching the page using urllib2.urlopen, all such href's containing 'semicolons' are truncated. For example the href http://travel.yahoo.com/p-travelguide-6901959-pune_restaurants-i;_ylt=