Eduardo A. Bustamante López added the comment:
I'm not sure what's the best approach here.
1. Avoid changes in the Lib, and document a work-around, which involves
installing an opener with the specific User-agent. The draw-back is that it
modifies the behaviour of urlopen() globally, so that change affects any
other call to urllib.request.urlopen.
2. Revert to the old way, using an instance of a FancyURLopener (or URLopener),
in the RobotFileParser class. This requires a modification of the Lib, but
allows us to modify only the behaviour of that specific instance of
RobotFileParser. The user could sub-class FancyURLopener, set the appropiate
version string.
I attach a script, tested against the ``default`` branch of the mercurial
repository. It shows the work around for python3.3.
----------
Added file: http://bugs.python.org/file27158/test.py
_______________________________________
Python tracker <rep...@bugs.python.org>
<http://bugs.python.org/issue15851>
_______________________________________
import urllib.robotparser
import urllib.request
opener = urllib.request.build_opener()
opener.addheaders = [('User-agent', 'MyUa/0.1')]
urllib.request.install_opener(opener)
rp = urllib.robotparser.RobotFileParser('http://localhost:9999')
rp.read()
_______________________________________________
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
