Hi, we have recently activated multi threading in our haproxy (version 1.8.14) and stumbled over the following problem:
we balance ftp via haproxy and after enabling multi threading, trying to transfer very small files via ftp sometimes failed, with log entries like this: Oct 20 22:45:58 vptest02 haproxy[11753]: 127.0.0.1:56631 [20/Oct/2018:22:45:58.787] ftp ftp/ftp 1/-1/0 0 CC 2/2/1/1/0 0/0 I could narrow down the problem to the option abortonclose. If enabled and the frontend receives the complete data transfer before fully establishing a connection to the backend the backend connection gets reseted and the line above is logged. Attached is a test haproxy configuration were i was able to reproduce the problem and a tcpdump showing the problem. I'm not sure if this is a bug or the documentation of the abortonclose option should be expanded. From the documentation I first thought that the option abortonclose only works in conjunction with mode http, because only the http protocol is mentioned. In tcp mode the question in my opinion is: "When is a connection aborted?". Thinking about it I came to the conclusion that the abortonclose can make sense on mode tcp too as long as the protocol used on the connection requires an answer from the server. But if that is the case then the documentation of abortonclose should explicitly mentioning that there are tcp based protocols with which it will produce errors because they don't expect an answer from the server. Regarding why it only happens with multi threading enabled: Could it be that a single threaded haproxy always opens the backend connection fully prior to closing the frontend connection and therefore avoided the problem altogether? But I'm no haproxy expert so I can be completely wrong and the issue is something completely differently. Please let me know if further information is needed to investigate this issue. Regards Reinhard Vicinus
global
cpu-map auto:1/1-4 0-3
nbproc 1
nbthread 4
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin expose-fd
listeners
stats timeout 30s
user haproxy
group haproxy
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
# An alternative list with additional directives can be obtained from
#
https://mozilla.github.io/server-side-tls/ssl-config-generator/?server=haproxy
ssl-default-bind-ciphers
ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3
defaults
log global
mode http
option httplog
option dontlognull
option abortonclose
timeout connect 5000
timeout client 50000
timeout server 50000
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
frontend ftp
bind 127.0.0.121:21
bind 127.0.0.121:10000-10250
default_backend ftp
mode tcp
backend ftp
fullconn 128
mode tcp
server ftp 10.138.3.245 check port 21
abortonclose.pcap
Description: application/vnd.tcpdump.pcap
