Re: Web page data and urllib2.urlopen

2009-08-07 Thread Piet van Oostrum
> Dave Angel (DA) wrote: >DA> Piet van Oostrum wrote: >>> >DA> But the raw page didn't have any javascript. So what about that original >DA> raw page triggered additional stuff to be loaded? >DA> Is it "user agent", as someone else brought out? And is there somewhere I

Re: Web page data and urllib2.urlopen

2009-08-07 Thread Dave Angel
Piet van Oostrum wrote: DA> But the raw page didn't have any javascript. So what about that original DA> raw page triggered additional stuff to be loaded? DA> Is it "user agent", as someone else brought out? And is there somewhere I DA> can read more about that aspect of thing

Re: Web page data and urllib2.urlopen

2009-08-07 Thread Piet van Oostrum
> Dave Angel (DA) wrote: >DA> Piet van Oostrum wrote: >>> >DA> If Mozilla had seen a page with this line in an appropriate place, it'd >DA> immediately begin loading the other page, at "someotherurl" But there's no >DA> such line. >>> >>> >DA> Next, I looked for javascript. The Moz

Re: Re: Web page data and urllib2.urlopen

2009-08-06 Thread Kushal Kumaran
On Fri, Aug 7, 2009 at 3:47 AM, Dave Angel wrote: > > > Piet van Oostrum wrote: >> >> >>> >>> DA> All I can guess is that it has something to do with "browser type" or >>> DA> cookies.  And that would make lots of sense if this was a cgi page. >>>  But >>> DA> the URL doesn't look like that, as it

Re: Re: Web page data and urllib2.urlopen

2009-08-06 Thread Dave Angel
Piet van Oostrum wrote: DA> If Mozilla had seen a page with this line in an appropriate place, it'd DA> immediately begin loading the other page, at "someotherurl" But there's no DA> such line. DA> Next, I looked for javascript. The Mozilla page contains lots of DA> javascript, b

Re: Web page data and urllib2.urlopen

2009-08-06 Thread Piet van Oostrum
> Dave Angel (DA) wrote: >DA> Massi wrote: >>> Hi everyone, I'm using the urllib2 library to get the html source code >>> of web pages. In general it works great, but I'm having to do with a >>> financial web site which does not provide the souce code I expect. As >>> a matter of fact if you

Re: Web page data and urllib2.urlopen

2009-08-06 Thread ryles
On Aug 5, 4:30 pm, Massi wrote: > Hi everyone, I'm using the urllib2 library to get the html source code > of web pages. In general it works great, but I'm having to do with a > financial web site which does not provide the souce code I expect. As > a matter of fact if you try: > > import urllib2

Re: Web page data and urllib2.urlopen

2009-08-05 Thread Dave Angel
Massi wrote: Hi everyone, I'm using the urllib2 library to get the html source code of web pages. In general it works great, but I'm having to do with a financial web site which does not provide the souce code I expect. As a matter of fact if you try: import urllib2 res = urllib2.urlopen("http:/

Web page data and urllib2.urlopen

2009-08-05 Thread Massi
Hi everyone, I'm using the urllib2 library to get the html source code of web pages. In general it works great, but I'm having to do with a financial web site which does not provide the souce code I expect. As a matter of fact if you try: import urllib2 res = urllib2.urlopen("http://www.marketwatc