At 1/18/2002 01:16 PM -0500, you wrote: >On 18 Jan 2002, Jeff Bearer wrote: > >I have a website that is being spidered by 1 host at in-opertune times, > >I'm trying to see if there is a way I can block the host in apache for a > >few hours of the day but allow it the rest of the day. > >Run a google on "robots.txt".
But remember that the robots.txt file is a guide for spiders; the spider can choose to respect or ignore those instructions. If you want to *block* the spider or if the spider is configured to ignore that file, you'll have to find some other way. -- Rodolfo J. Paiz [EMAIL PROTECTED] _______________________________________________ Redhat-list mailing list [EMAIL PROTECTED] https://listman.redhat.com/mailman/listinfo/redhat-list
