This thread describes something that I found by accident:
http://www.webmasterworld.com/msn_microsoft_search/4161750.htm
I have a customer using web connect and they noticed (due to the
original developer's error) that the Bing search bot was hitting their
site hundreds of times a day. We blocked the IP blocks found in the web
connect request log. This week they started coming from a different
block and after blocking that range they increased to hundreds again
from the blocked range. They are not respecting the robots.txt either.
They are getting a response that says they don't have access to the site
because they are not logged in. It looks like if you are indexed they
go there even if you block them with a robots.txt after the fact.
What's up with these guys?
--
Jeff
---------------
Jeff Johnson
[email protected]
(623) 582-0323
www.san-dc.com
www.arelationshipmanager.com
--- StripMime Report -- processed MIME parts ---
multipart/alternative
text/plain (text body -- kept)
text/html
---
_______________________________________________
Post Messages to: [email protected]
Subscription Maintenance: http://mail.leafe.com/mailman/listinfo/profox
OT-free version of this list: http://mail.leafe.com/mailman/listinfo/profoxtech
Searchable Archive: http://leafe.com/archives/search/profox
This message: http://leafe.com/archives/byMID/profox/[email protected]
** All postings, unless explicitly stated otherwise, are the opinions of the
author, and do not constitute legal or medical advice. This statement is added
to the messages for those lawyers who are too stupid to see the obvious.
