On 18:40 18 Jan 2002, Rodolfo J. Paiz <[EMAIL PROTECTED]> wrote: | At 1/18/2002 01:16 PM -0500, you wrote: | >On 18 Jan 2002, Jeff Bearer wrote: | > >I have a website that is being spidered by 1 host at in-opertune times, | > >I'm trying to see if there is a way I can block the host in apache for a | > >few hours of the day but allow it the rest of the day. | > | >Run a google on "robots.txt". | | But remember that the robots.txt file is a guide for spiders; the spider | can choose to respect or ignore those instructions. If you want to *block* | the spider or if the spider is configured to ignore that file, you'll have | to find some other way.
True, but nonetheless it's the first thing to try. Always try the cooperative (and "standard") approach before coercion. -- Cameron Simpson, DoD#743 [EMAIL PROTECTED] http://www.zip.com.au/~cs/ Anarchy is not lack of order. Anarchy is lack of ORDERS. _______________________________________________ Redhat-list mailing list [EMAIL PROTECTED] https://listman.redhat.com/mailman/listinfo/redhat-list
