over the last 90 days or so, we seem to have been plauged with a new breed of bots scanning our web pages, and they are not just indexing our web pages I don't mind that, but they are ignoring our robots.txt and are mirroring anything apache2 can reach, including stuff thats there but not reachable by a normal browser just looking around and clicking on links. Its annoying as hell and when you're out in the pucker-brush on a 10 megabit ADSL, eats up ones available upload bandwidth of about 275kbytes/s. According to my cable billing, these A-H's used over 100Gb of my bandwidth in Nov 2019. That describes in printable language as a DDOS in my vocabulary.
So I asked a few questions and wrote some little 2-3 line scripts after putting a tail on /var/lib/httpd/other_vhosts_access.log, which logs enough info you can generally identify the bots with it. I have since have generated 49 iptables rules that have blocked 99% of them. Those scripts I've placed in /etc/iptables and are owned by root. To start iptables after a reboot:You might run this one first one from /etc/rc.local root@coyote:iptables$ cat start-iptables #!/bin/bash cd /etc/iptables iptables-restore <rules.v4 To add a new rule, covering that whole 256 address block because they seem to have a random address, changed about weekly, in that block: root@coyote:iptables$ cat iptables-add #!/bin/bash iptables -I INPUT -s add.ress.to.block/24 -j DROP Substituting the address of the offender for add.ress.to.block in the last tine above. to save the rules: root@coyote:iptables$ cat iptables-saveem #!/bin/bash iptables-save >rules.v4 To see what you've got so far: root@coyote:iptables$ cat iptables-status #!/bin/bash iptables -L -nv --line-numbers Which will output the rules in effect plus the hits accumulated in this uptime so far, in this format: lnum hits bytes fate 24 846 50760 DROP all -- * * 66.249.64.0/24 0.0.0.0/0 Be my guest folks, reclaim the net, we are paying for the bandwidth these jerks are burning up. Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) If we desire respect for the law, we must first make the law respectable. - Louis D. Brandeis Genes Web page <http://geneslinuxbox.net:6309/gene>
