[issue13281] Make robotparser.RobotFileParser ignore blank lines

2020-11-17 Thread Irit Katriel
Change by Irit Katriel : -- versions: +Python 3.10, Python 3.9 -Python 3.3 ___ Python tracker ___ ___ Python-bugs-list mailing list

[issue13281] Make robotparser.RobotFileParser ignore blank lines

2011-11-12 Thread Terry J. Reedy
Terry J. Reedy added the comment: My suggested doc change is how to change the doc along with the patch. -- ___ Python tracker ___ __

[issue13281] Make robotparser.RobotFileParser ignore blank lines

2011-11-12 Thread Éric Araujo
Éric Araujo added the comment: First, I’d like to remind that the robots spec is not an official Internet spec backed up by an official body. It’s also not as important as (say) HTTP parsing. For this bug, IMO the guiding principle should be Postel’s Law. What harm is there in being more l

[issue13281] Make robotparser.RobotFileParser ignore blank lines

2011-11-02 Thread Ezio Melotti
Changes by Ezio Melotti : -- nosy: -osvenskan ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyth

[issue13281] Make robotparser.RobotFileParser ignore blank lines

2011-11-02 Thread Petri Lehtinen
Petri Lehtinen added the comment: > My current suggested doc change is to replace the sentence quoted at the top > with Sounds good to me. -- ___ Python tracker ___ __

[issue13281] Make robotparser.RobotFileParser ignore blank lines

2011-10-31 Thread Terry J. Reedy
Terry J. Reedy added the comment: The robotparser is currently doing exactly what it is documented as doing. 20.9. urllib.robotparser — Parser for robots.txt says "For more details on the structure of robots.txt files, see http://www.robotstxt.org/orig.html."; (Since there are no previous deta

[issue13281] Make robotparser.RobotFileParser ignore blank lines

2011-10-31 Thread Ezio Melotti
Ezio Melotti added the comment: If it's added it should be a versionchanged, not a versionadded. I'm also not entirely sure this should be considered a new feature and don't see the point of having a strict mode. IMHO robotparser should honor what the robots.txt files say, and not doing so be

[issue13281] Make robotparser.RobotFileParser ignore blank lines

2011-10-30 Thread Terry J. Reedy
Terry J. Reedy added the comment: Since following the spec is not a bug, this issue is a feature request for 3.3. I agree with just being lenient with no added parameter. Perhaps that should be mentioned in the doc with (or in) a version-added note. Senthil: does doc standard allow something