Once upon a time (actually it was more like Fri, 23 Nov 2001 17:32:04 +0000 (UTC)), Martin WHEELER <[EMAIL PROTECTED]> wrote:
> Is anyone else having problems with the robot from > > openfind.com.tw > > -- an intrusive, irritating, hard-to-get-rid-of crawler that completely > paralyses my system *every day*? > > Despite what I put in any robots.txt, this one disregards all rules and > just jams up my system, downloading every damn' thing in sight. > Mails to the owners are totally disregarded. Have you actually seen: http://www.openfind.com.tw/robot.html It talks about the robot and how to get it to stop accessing your site. Hereward
msg04113/pgp00000.pgp
Description: PGP signature
