IIUC robots.txt will allow you to ‘deny /; allow /index.html’
From: pydotorg-www [mailto:pydotorg-www-bounces+tritium-list=sdamon....@python.org] On Behalf Of Skip Montanaro Sent: Tuesday, November 1, 2016 8:40 AM To: pydotorg-www@python.org Subject: Re: [pydotorg-www] Steering search engines and users away from 3.0/3.1 docs? Tweaking robots.txt seems like the simplest route. It would be nice if /3.[01]/index.html where visible through search engines but not anything underneath. I don't recall if robots.txt is a sharp enough tool to make that distinction. Following my original thought a bit further, I wondered how far down in the search results the mainline /3/ docs would be, so I modified my search to "python timeit site:docs.python.org <http://docs.python.org> ". The answer for me, at least, seems to be, "almost not on the front page." In fact, the 2.5 timeit reference is right after the 3.0 reference. 3.1 is a bit further down, then the v3 version of the quick stdlib tour is the last hit on page one. If someone's going to mess with robots.txt, maybe cast the same spell for 2.[0-5] (and 6?) as you do for 3.0 and 3.1. Skip
_______________________________________________ pydotorg-www mailing list pydotorg-www@python.org https://mail.python.org/mailman/listinfo/pydotorg-www
