Not a tomcat issue but I can't say enough good things about cloudflare which we use. Easy to configure WAF so the site is locked down to cloudflare only and the rest of the world has to come through them where each keystroke is inspected. They have a number of preconfigured rules that mostly just work right out of the box. Various plans ranging from free and up. We did this years ago on every site.
On Thu, Jun 27, 2024 at 11:03 AM Christopher Schultz < ch...@christopherschultz.net> wrote: > James, > > On 6/24/24 17:57, James H. H. Lampert wrote: > > On 6/24/24 12:03 PM, Tim Funk wrote: > >> Conversely, this is a good time for the developers to review > >> their server logging and tune it to be less verbose for these > >> normal exceptions. As well as implementing logging frameworks > >> and logging at the appropriate level (fatal through debug) > > > > Thanks for your thoughts on the subject. > > > > The primary problem isn't that the catalina.out file is enormous, nor > > that the log messages are too verbose (if anything, they're not verbose > > enough: a timestamp would help) ; it's that while the random filenames > > were being tried, in quick succession, the CPU usage of the JVM job > > skyrocketed, and legitimate traffic was probably having trouble getting > in. > > Hmm. Are you getting requests for the same (missing) resource over and > over again, or many different ones? I suspect different, since attackers > have a huge dictionary of "telltale resources" to identify a service as > running one of many known-vulnerable services (e.g. Wordpress, JIRA, etc.). > > Why aren't you seeing the source-IP in your own logs? If you are using a > reverse-proxy and always getting the IP of the proxy, you should be able > to (a) configure the reverse-proxy to provide its own IP to you and > configure Tomcat to use that for logging and/or (b) look-up the same > request in the proxy logs. Once you have that, you can ban. > > Banning can be done in many ways. > > One way would be just by-hand. If you see a huge number of requests from > one IP, just ban it. If it's more than one IP, then you get to play > whack-a-mole and automated solutions would be better. > > Be careful with fail2ban (which I like a lot, but I've never used it > with HTTP logs): you can easily ban people quickly who aren't actually > attacking you. If you use fail2ban (or similar), I would be very > conservative about its configuration. Basically, you choose the number > of flags that have to be thrown per-IP per-time-interval. So something > like "100 404s in a minute per-IP" would be conservative. > > fail2ban is definitely available on Linux and probably also BSD. I think > you are on AS/400 or something like that, right? And of course you'd > want to ban as far up the chain as possible, at the reverse-proxy, TLS > terminator, or load-balancer. I'm not sure exactly what options you have > in that environment. fail2ban works directly with the OS's firewall, so > on Linux that's iptables and on BSD that's generally ipfilter. If AS/400 > has a built-in firewall then it's possible fail2ban has a build for it. > > Tim's suggestion of a WAF is a good one. I've tried to configure > mod_security for Apache httpd and it's got like 100M rules and it > immediately banned every one of our developers when we used it > out-of-the-box. It is SUPER PARANOID about everything. If you are > considering using mod_security, I would (a) run it only in a development > / testing environment and (b) run it in "audit mode" for a week or so > and look at all the complaints it makes. I have had to specifically > disable a LARGE number of checks to make it even partially usable. > > -chris > > --------------------------------------------------------------------- > To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org > For additional commands, e-mail: users-h...@tomcat.apache.org > >
