[issue4108] robotparser.py fail when more than one User-Agent: * is present

Georg Brandl added the comment: Thanks for the patch, fixed in r83238. -- nosy: +georg.brandl resolution: -> fixed status: open -> closed ___ Python tracker ___

[issue4108] robotparser.py fail when more than one User-Agent: * is present

mARK added the comment: this looks like a good fix. i've put it into my own copy. -- nosy: +mbloore ___ Python tracker ___ ___ Python

[issue4108] robotparser.py fail when more than one User-Agent: * is present

New submission from taskinoor hasan sajid <[EMAIL PROTECTED]>: Check the robots.txt file from mathworld. --> http://mathworld.wolfram.com/robots.txt It contains 2 User-Agent: * lines. >From http://www.robotstxt.org/norobots-rfc.txt "These name tokens are used in User-agent lines in /robots.tx