Writing a class which essentially spiders a site and saves the files
locally. On a URLError exception, it sleeps for a second and tries again (on
404 it just moves on). The relevant bit of code, including the offending
method:
class Handler(threading.Thread):
def __init__(self, url):
threading.Thread.__init__(self)
self.url = url
def save(self, uri, location):
try:
handler = urllib2.urlopen(uri)
except urllib2.HTTPError, e:
if e.code == 404:
return
else:
print "retrying %s (HTTPError)" % uri
time.sleep(1)
self.save(uri, location)
except urllib2.URLError, e:
print "retrying %s" % uri
time.sleep(1)
self.save(uri, location)
if not os.path.exists(os.path.dirname(location)):
os.makedirs(os.path.dirname(location))
file = open(location, "w")
file.write(handler.read())
file.close()
...
But what I am seeing is that after a retry (on catching a URLError
exception), I see bunches of "UnboundLocalError: local variable 'handler'
referenced before assignment" errors on line 38, which is the
"file.write(handler.read())" line..
What gives?
--
Wells Oliver
we...@submute.net
--
http://mail.python.org/mailman/listinfo/python-list
