On Wed, Mar 19, 2003 at 10:20:56AM -0800, Gabriela Valdes wrote: > As you may know, Google's mission is to deliver the best search > experience on the Internet by making the world's information universally > accessible and useful. Currently, our Google.com site is within the top > 10 sites in all major markets worldwide with over 200 million searches > per day. > > We believe that www.debian.org is a great site and have discovered that > Google is currently blocked from crawling your site by the robot.txt > that is on your site. I believe we can drive a lot of traffic and > awareness to your organization and would like to find a mutually > beneficial way to work together.
How's that? http://www.debian.org/robots.txt says only: User-agent: * Disallow: /security/ Disallow: I'm not sure why we ban /security/, but otherwise it should be perfectly possible to crawl the remaining 773 MB of www.debian.org... -- 2. That which causes joy or happiness.
