On Thursday 03 December 2020 08:07:33 john doe wrote: > On 12/3/2020 1:35 PM, Gene Heskett wrote: > > I've had it with a certain bot that that ignore my robots.txt and > > proceeds to mirror my site, several times a day, burning up my > > upload bandwidth. They've moved it to 5 different addresses since > > midnight. > > > > I want to nail the door shut on the first attempted access by these > > AH's. > > > > Does anyone have a ready made script that can watch my httpd "other" > > log, and if a certain name is at the end of the line, grabs the ipv4 > > src address as arg3 of the line, and applies it to iptables DROP > > rules? > > > > Or do I have to invent a new wheel for this? > > > > Basic rules that simplify it somewhat. > > > > 1. this is ipv4 only country and not likely to change in the future > > decade. > > > > 2. the list of offending bot names will probably never go beyond 50, > > if that many. 5 would be realistic. > > > > 3. the src address in the log is at a fixed offset, obtainable with > > the bash MID$ but the dns return will need some acrobatics involving > > the bash RIGHT$ function. > > > > 4. it should track the number of hits, and after so many in a /24 > > block, autoswitch to a /16 block in order to keep the rules file > > from exploding. > > Is that not the same question you asked a while back, I then suggested > 'fail2ban' or using ip/nftables own capabilities? > Yes John. But explain to me what fail2ban is sopposed to do?
Its running, but has failed to ban anything no matter what sort of 403's I return. Fail2ban has been running here for years, and in just sits there doing nothing, so if its as great a swiss army knife as others claim it to be, lets either make it work, or quit recommending it. I need something I can feed with a tee off the tail output, detect that it is one of the offending bots by name, and if so, apply its ipv4 address to an iptables DROP rule. > It looks to me like you are making your life way much harder than it > should be. Fine, now show me how to make fail2ban do something usefull. I just rebooted because of a drive failure and found it couldn't be found running by htop, so I started it. Now make it do something usefull if its so great. iptables does work but I have to manually pick the addresses out of the log in order to put them into the rules. They move these bots around 2 or more times a month to get around people like me who do use something like iptables, so Ideally I should nuke the rules file about monthly and restart a new compilation by feeding this script with the last 5000 lines of the "other" log. And leave the tail active to feed new hits into this script one line at a time as they occur. Thank you. > -- > John Doe Cheers, Gene Heskett -- "There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order." -Ed Howdershelt (Author) If we desire respect for the law, we must first make the law respectable. - Louis D. Brandeis Genes Web page <http://geneslinuxbox.net:6309/gene>
