On May 27, 1:50 pm, Jeff McNeil <j...@jmcneil.net> wrote: > On May 27, 12:29 pm, powah <wong_po...@yahoo.ca> wrote: > > > > > > > I want to download all mib files from the web > > page:http://www.juniper.net/techpubs/software/junos/junos94/swconfig-net-m... > > > All mib filenames are of this format:www.juniper.net/techpubs... .txt > > > I write this program but has the following error. > > Please help. > > Thanks. > > > [code] > > #!/usr/bin/env python > > import urllib2,os,urlparse > > url="http://www.juniper.net/techpubs/software/junos/junos94/swconfig- > > net-mgmt/juniper-specific-mibs-junos-nm.html#jN18E19" > > page=urllib2.urlopen(url) > > f=0 > > links=[] > > data=page.read().split("\n") > > for item in data: > > if "www.juniper.net/techpubs" in item: > > httpind=item.index("www.juniper.net/techpubs") > > item=item[httpind:] > > #print "item " + item > > ind=item.index("<") > > links.append(item[:ind]) #grab all links > > # download all links > > for link in links: > > print "link " + link > > filename=link.split("/")[-1] > > print "downloading ... " + filename > > u=urllib2.urlopen(link) > > p=u.read() > > open(filename,"w").write(p) > > [/code] > > > $ ~/python/downloadjuniper.py > > linkwww.juniper.net/techpubs/software/junos/junos94/swconfig-net-mgmt/mib... > > downloading ... mib-jnx-user-aaa.txt > > Traceback (most recent call last): > > File "/home/powah/python/downloadjuniper.py", line 20, in ? > > u=urllib2.urlopen(link) > > File "/usr/lib/python2.4/urllib2.py", line 130, in urlopen > > return _opener.open(url, data) > > File "/usr/lib/python2.4/urllib2.py", line 350, in open > > protocol = req.get_type() > > File "/usr/lib/python2.4/urllib2.py", line 233, in get_type > > raise ValueError, "unknown url type: %s" % self.__original > > ValueError: unknown url > > type:www.juniper.net/techpubs/software/junos/junos94/swconfig-net-mgmt/mib... > > > $ python > > Python 2.4.4 (#1, Oct 23 2006, 13:58:00) > > [GCC 4.1.1 20061011 (Red Hat 4.1.1-30)] on linux2 > > Type "help", "copyright", "credits" or "license" for more information. > > > My computer is FC6 linux. > > There's only a couple dozen of them, right-click->Save As. I'm sure > Juniper would appreciate that much more than an automated crawler. > > As far as your ValueError is concerned, consider that > 'www.juniper.com'doesn't start with a protocol specification when > passed into urllib2.urlopen. > > -Jeff > mcjeff.blogspot.com
Juniper's web page is simple, I am learning python so as to download files from more complex web page and do other things as well. -- http://mail.python.org/mailman/listinfo/python-list
