On Wednesday 26 February 2020 04:21:09 Jonas Smedegaard wrote: > Quoting Gene Heskett (2020-02-26 09:57:51) > > > over the last 90 days or so, we seem to have been plauged with a new > > breed of bots scanning our web pages, and they are not just indexing > > our web pages I don't mind that, but they are ignoring our > > robots.txt and are mirroring anything apache2 can reach, including > > stuff thats there but not reachable by a normal browser just looking > > around and clicking on links. Its annoying as hell and when you're > > out in the pucker-brush on a 10 megabit ADSL, eats up ones available > > upload bandwidth of about 275kbytes/s. > > Download "eating" upload on ADSL might be due to bufferbloat: > https://www.bufferbloat.net/projects/bloat/wiki/What_can_I_do_about_Bu >fferbloat/ > Maybe, but I can still go read the news IF my browser can sneak a page request in between upload packets when gkrellm is showing 300k of upload at the instant. > And I'm subscribed Jonas, no need to reply all.
> - Jonas Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) If we desire respect for the law, we must first make the law respectable. - Louis D. Brandeis Genes Web page <http://geneslinuxbox.net:6309/gene>
