2011/12/5 Chí-Thanh Christopher Nguyễn :
> Alec Warner schrieb:
>>> Seriously, what do we gain from crawlers accessing sources.gentoo.org? I
>>> cant
>>> really remember seeing it once in a google query result...
>>
>> We want the site searchable.
>
The majority of the expensive requests are
Alec Warner schrieb:
>> Seriously, what do we gain from crawlers accessing sources.gentoo.org? I
>> cant
>> really remember seeing it once in a google query result...
>
> We want the site searchable.
>>> The majority of the expensive requests are related to package.mask and
>>> use.local.desc q
On Mon, Dec 5, 2011 at 3:48 AM, Andreas K. Huettel wrote:
>
> Seriously, what do we gain from crawlers accessing sources.gentoo.org? I cant
> really remember seeing it once in a google query result...
We want the site searchable.
>
> Possibly it would not even be required to deny all requests,
Seriously, what do we gain from crawlers accessing sources.gentoo.org? I cant
really remember seeing it once in a google query result...
Possibly it would not even be required to deny all requests, but just deny
everything related to ancient history...
> Hello,
>
> For a while sources.gento
Hello,
For a while sources.gentoo.org has been puttering along and its health
has slowly declined. We migrated it to some newer shiny hardware in an
attempt to mitigate the problem but that did not pan out. 90% (or
more) of sources.gentoo.org traffic is crawler bots and not actual
humans. That bei
