I can't tell if this is possibly related to this closed issue: [ https://issues.apache.org/jira/browse/TS-1935 | https://issues.apache.org/jira/browse/TS-1935 ]
Yesterday I had a DoS incident where a single client generating 30-50 requests per second caused my ATS instance to fail. The failure was unusual in that it was not directly load related, but rather that a large percentage of requests to ATS would fail in the TLS handshake with the client. It appears that some interaction between misbehaving ATS and the misbehaving client was to blame. Restarting ATS did not resolve the problem, and I had to block the offending IP address. I do not believe this was an intentional DoS attack. Investigating ATS logs, I have thousands of requests logged as: 1556735491.734 26 71.255.116.97 ERROR_UNKNOWN(90)/000 0 GET https://wordpress.internal.convivian.com/participate/join-a-core/medical/ - EMPTY/wordpress.internal.convivian.com - 1556735491.759 7 71.255.116.97 ERROR_UNKNOWN(90)/000 0 GET https://wordpress.internal.convivian.com/participate/join-a-core/medical/ - EMPTY/wordpress.internal.convivian.com - 1556735491.829 69 71.255.116.97 ERROR_UNKNOWN(90)/000 0 GET https://wordpress.internal.convivian.com/participate/join-a-core/medical/ - EMPTY/wordpress.internal.convivian.com - [etc.] These correspond to successful requests on the origin server: www.fireflyartscollective.org:443 71.255.116.97 - - [01/May/2019:14:31:31 -0400] "GET /participate/join-a-core/medical/ HTTP/1.1" 200 16296 "https://www.fireflyartscollective.org/participate/join-a-core/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0" www.fireflyartscollective.org:443 71.255.116.97 - - [01/May/2019:14:31:31 -0400] "GET /participate/join-a-core/medical/ HTTP/1.1" 200 16296 "https://www.fireflyartscollective.org/participate/join-a-core/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0" www.fireflyartscollective.org:443 71.255.116.97 - - [01/May/2019:14:31:31 -0400] "GET /participate/join-a-core/medical/ HTTP/1.1" 200 16296 "https://www.fireflyartscollective.org/participate/join-a-core/"; "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0" [etc.] Outside of this odd DoS, ERROR_UNKNOWN(90)/000 does not appear to be uncommon. What does this indicate? # traffic_logcat squid.blog_serviceproxy.convivian.com.20190501.00h00m50s-20190502.00h00m02s.old | cut -f 4 -d ' ' | sort | uniq -c | sort -rn | head 130628 TCP_MISS/200 20156 TCP_MISS/301 13009 TCP_MISS/302 7627 ERROR_UNKNOWN(90)/000 6845 TCP_IMS_MISS/302 1949 TCP_MISS/404 1139 TCP_MISS/501 1022 TCP_MISS/207 836 TCP_MISS/303 813 TCP_IMS_MISS/200 I am unable to reproduce the DoS, but I do see many routine ERROR_UNKNOWN(90)/000 in the logs. Regards, --Jered
