[Christof] > Personally, I am seeing gptbot crawling at a rate of up to about 1 > request per second.
[Joerg] > My own web sites see moderate scraper traffic, but they don't have a > large site graph either. On various other sites like anonhg.n.o, the > main Mercurial bug tracker etc we have been observing persistent high > load. Yeah, and I'm seeing some crawlers, but nothing even I would call high load, much less what something like cvsweb or anonhg would...even though I have, from a Web client's perspective, *lots* of source trees, including many (slightly different) NetBSD source trees. Makes me wonder what the relevant difference is. Border router blocks? Lack of HTTPS support? They don't recognize the interface I present? They just haven't noticed me yet? /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTML mo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
