Gene Heskett wrote: > over the last 90 days or so, we seem to have been plauged with a new > breed of bots scanning our web pages, and they are not just indexing our > web pages I don't mind that, but they are ignoring our robots.txt and > are mirroring anything apache2 can reach, including stuff thats there > but not reachable by a normal browser just looking around and clicking > on links. Its annoying as hell and when you're out in the pucker-brush > on a 10 megabit ADSL, eats up ones available upload bandwidth of about > 275kbytes/s. According to my cable billing, these A-H's used over 100Gb > of my bandwidth in Nov 2019. That describes in printable language as a > DDOS in my vocabulary. > > So I asked a few questions and wrote some little 2-3 line scripts after > putting a tail on /var/lib/httpd/other_vhosts_access.log, which logs > enough info you can generally identify the bots with it. > > I have since have generated 49 iptables rules that have blocked 99% of > them.
If you find yourself needing to add lots more rules, you might want to generate a "set" instead of individual rules: http://ipset.netfilter.org/ https://www.linuxjournal.com/content/advanced-firewall-configurations-ipset might be useful. -dsr-
