Add the line User-agent: * Disallow: /BugsDisallow: /Bugs
/Disallow: /Bugs/ On 13-Dec-97 [EMAIL PROTECTED] wrote: > Does this mean that they are finding it through our own > WWW archives of the mailing lists? Yes. Search indexes like Yahoo index the whole site. > Do we need to add something there to keep them from being indexed? > > Bruce > Add the line: Disallow: /Lists-Archives/ to the robots.txt in the root directory of the web site. It currently reads: User-agent: * Disallow: /Bugs/ This will stop the "robots" from the search engines indexing that part of the website. The only problem is that it won't clear what they previously have indexed. Also, the change needs to be made to all site that mirror the mailing lists. ---------------------------------- Please do not look directly into laser with remaining eye. E-Mail: Mark Ciciretti <[EMAIL PROTECTED]> Date: 14-Dec-97 Time: 21:04:01 ---------------------------------- -- TO UNSUBSCRIBE FROM THIS MAILING LIST: e-mail the word "unsubscribe" to [EMAIL PROTECTED] . Trouble? e-mail to [EMAIL PROTECTED] .
