-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
OK, I've now been 24 hours without a hit, so I'm presuming I've got rid
of all the crawlers.
Thanks for all the help and advice from both lists.
Resume:
- - the openfind.com(.tw) 'bots don't respect the norobots conventions, so
your robots.txt is us
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
OK, I've now been 24 hours without a hit, so I'm presuming I've got rid
of all the crawlers.
Thanks for all the help and advice from both lists.
Resume:
- - the openfind.com(.tw) 'bots don't respect the norobots conventions, so
your robots.txt is u
The best way would be to block it at your router with an access list.
Blocking it at the box is ok too but that takes a little bit of your
resources. And you have to do it on each box on your network you want
protected. The router block will protect your entire network in one fell
swoop and cost
The best way would be to block it at your router with an access list.
Blocking it at the box is ok too but that takes a little bit of your
resources. And you have to do it on each box on your network you want
protected. The router block will protect your entire network in one fell
swoop and cost
Martin WHEELER <[EMAIL PROTECTED]> writes:
> Is anyone else having problems with the robot from
> openfind.com.tw
> -- an intrusive, irritating, hard-to-get-rid-of crawler that completely
> paralyses my system *every day*?
Nope. How does it paralyse you, anyway?
> Despite what I put in any
Is anyone else having problems with the robot from
openfind.com.tw
-- an intrusive, irritating, hard-to-get-rid-of crawler that completely
paralyses my system *every day*?
Despite what I put in any robots.txt, this one disregards all rules and
just jams up my system, downloading every damn
Hi all
On Fri, Nov 23, 2001 at 06:13:29PM +, Tim Haynes wrote:
> Martin WHEELER <[EMAIL PROTECTED]> writes:
>
> > Is anyone else having problems with the robot from
> > openfind.com.tw
> > -- an intrusive, irritating, hard-to-get-rid-of crawler that completely
> > paralyses my system *e
On Sun, 25 Nov 2001, Dave Watkins wrote:
> I'm sure it's been said before but why not just configure iptables to drop
> the packets from 139.175.250.23?
>
> Then it CAN'T connect
Already done.
It's just taking a little while to kill off 16 different crawlers, which
I hadn't appreciated was what w
I'm sure it's been said before but why not just configure iptables to drop
the packets from 139.175.250.23?
Then it CAN'T connect
At 07:34 PM 11/24/01 +, Martin WHEELER wrote:
>On Sat, 24 Nov 2001, Hereward Cooper wrote:
>
> > > Despite what I put in any robots.txt, this one disregards all
Once upon a time (actually it was more like Fri, 23 Nov 2001 17:32:04 +
(UTC)),
Martin WHEELER <[EMAIL PROTECTED]> wrote:
> Is anyone else having problems with the robot from
>
> openfind.com.tw
>
> -- an intrusive, irritating, hard-to-get-rid-of crawler that completely
> paralyses my
I'm sure it's been said before but why not just configure iptables to drop
the packets from 139.175.250.23?
Then it CAN'T connect
At 07:34 PM 11/24/01 +, Martin WHEELER wrote:
On Sat, 24 Nov 2001, Hereward Cooper wrote:
> > Despite what I put in any robots.txt, this one disregards all rules a
On Sat, 24 Nov 2001, Hereward Cooper wrote:
> > Despite what I put in any robots.txt, this one disregards all rules and
> > just jams up my system, downloading every damn' thing in sight.
> > Mails to the owners are totally disregarded.
>
> Have you actually seen:
> http://www.openfind.com.tw/robo
Once upon a time (actually it was more like Fri, 23 Nov 2001 17:32:04 +
(UTC)),
Martin WHEELER <[EMAIL PROTECTED]> wrote:
> Is anyone else having problems with the robot from
>
> openfind.com.tw
>
> -- an intrusive, irritating, hard-to-get-rid-of crawler that completely
> paralyses my s
On Fri, 23 Nov 2001, Michael Blickenstorfer wrote:
> But you can try...
Yeah. I have been. For about ten days now. :(
[BTW, my mis-attribution of origin was deliberate.
This Bad Attitude 'Bot has *really* got up my nose.]
But by applying *all* the useful suggestions made on this list (tha
Hi all
On Fri, Nov 23, 2001 at 06:13:29PM +, Tim Haynes wrote:
> Martin WHEELER <[EMAIL PROTECTED]> writes:
>
> > Is anyone else having problems with the robot from
> > openfind.com.tw
> > -- an intrusive, irritating, hard-to-get-rid-of crawler that completely
> > paralyses my system *ev
Martin WHEELER <[EMAIL PROTECTED]> writes:
> Is anyone else having problems with the robot from
> openfind.com.tw
> -- an intrusive, irritating, hard-to-get-rid-of crawler that completely
> paralyses my system *every day*?
Nope. How does it paralyse you, anyway?
> Despite what I put in any
Is anyone else having problems with the robot from
openfind.com.tw
-- an intrusive, irritating, hard-to-get-rid-of crawler that completely
paralyses my system *every day*?
Despite what I put in any robots.txt, this one disregards all rules and
just jams up my system, downloading every damn'
17 matches
Mail list logo
