[EMAIL PROTECTED] wrote: > On Wed, 14 Dec 2005, Angus Leeming wrote: >> http://validator.w3.org/checklink/ > > Hmm... I have to disable 'robots.txt', and it seems awfully slow. Thanks > anyway, I'll keep it in mind while looking at other options.
You can make exceptions in robots.txt for exactly this client. See http://validator.w3.org/docs/checklink.html The slowness is on purpose: Otherwise the linkchecker could be misused for DoS attacks. Georg
