Hi,
HAProxy 3.0.9 was released on 2025/03/20. It added 78 new commits
after version 3.0.8.
The main fixes are almost the same as the ones for 3.1.6, with a bunch
of more minor fixes that had been diluted into 3.1.4 and 3.1.5:
- in API issue in the applets could have resulted in some shutdown or
error conditions to be missed in the future, so as a prevention it
was fixed. Turns out, after fixing this, it uncovered a bug in the
CLI's "_getsocks" handler that was causing an infinite loop during
reloads, and another one in the SPOE applet where the appled would
never shut down (neither appeared in a released version), and these
bug were also fixed.
- the shorter watchdog delay in 3.1 allows to print a warning revealed
that we could sometimes deadlock between a thread dump (e.g. as called
by a stuck warning) and a panic. That's not cool because it could end
up with a process that spins forever instead of dying.
- reloads that transfer listening sockets to the new worker process could
make the older worker consume a lot of CPU for no apparent reason for
the time it remained present. The cause was that these FDs were
registered in epoll and when a new connection arrived to the new
process, the old one would also be notified without being able to
unregister it since already closed (well-known epoll pitfall). Now
these FDs are properly unregistered after being transfered so it's
possible that some users with long-running old processes will observe
a lower CPU usage on these old processes.
- a BUG_ON() could be triggered when using filters with no http_payload
callback.
- a bug in htx_xfer_blks() could result in occasionally transfering more
blocks than requested on 32-bit platforms.
- some TLSv1.3 signature algorithms were not recognized by the
ClientHello parser which was written before TLSv1.3. The ones that
were not correctly supported were based on RSA-PSS and would have
resulted in presenting a possibly wrong certificate when both RSA and
ECDSA ones were present for the same SNI.
- disabling the send-proxy-v2 feature on a "server" line after some fields
had been enabled in the defaults section would result in an attempt to
send a PROXY-v1 line because the presence of any field was tested to
decide to send the PROXY header.
- "show threads" and the watchdog warnings are using signals to dump the
streams' contents. However there was an indirect pool_alloc() performed
there when retrieving the stream client's source address, and if the
stream had been interrupted inside some pool functions, the re-entrance
could corrupt them and cause random crashes later. It's more visible in
3.1 than older versions due to the watchdog warnings that are non-fatal.
Now the address will be displayed only if it can safely be retrieved.
- similar to the above, "show threads" or a watchdog warning firing while
updating the stream's call_rate frequency counter could cause a deadlock
when trying to display that same frequency counter which would then be
locked. Now only an estimate of the call rate will be reported using a
non-blocking call, which can exceptionally be inaccurate but we don't
care since it's reported only to help spot suspicious streams.
- similar to the above, "show threads" or a watchdog warning firing while
reading the local time could result in a deadlock due to the libc using
a lock internally. Now the haproxy will not attempt to display the local
time from a signal handler.
- a startup race can randomly affect the setting of CPU affinity of
certain threads during boot, if the thread starts faster than the
parent updates the pthread identifier upon return. When it happens,
the affinity setting will often be silently ignored, but on some libs
such as musl, it can also result in a segfault at boot.
- a use-after-free condition was occasionally possible in Lua applets
handling CLI keywords, causing random crashes. It was apparently
difficult to trigger and did apparently not happen before 3.0.
- a possible crash was possible due to an assert condition on the purge
of QUIC streams depending on the ordering of received RESET_STREAM and
STOP_SENDING frames.
- SPOE applets could be woken in loops during stopping, thus eating a lot
of CPU until the process stopped. Note that this only concerns versions
3.0 and earlier.
And there were a few other less important ones among which:
- root (or suitable capability) was required for namespaces usage
even in frontend, which is not logical as it's only needed to attach
at run time for backends.
- a few isses on the peers section parser and config consistency checker
possibly causing issues or even a segfault.
- h2: leading and trailing spaces are now properly trimmed in header
values.
- servers attached to a ring wouldn't wait before trying to reconnect,
causing connection storms in case of a server failure.
- the H1 mux and general streams were made more robust against dumps
from a signal handler, by not keeping unallocated pointers visible
and making sure certain objects still exist before being displayed.
- the h3 mux will no longer forget to send the FIN when the server
responds before the end of the request. Without this, a client could
remain stuck waiting for it.
- the age calculation in the "show sess" output was often wrong for
streams just created (e.g. "99d"). This was confusing because grepping
on "age" is something common to detect stuck connections. Speaking of
"show sess", it now supports filtering by frontend/backend/server in
order to only focus on specific points.
- a few minor memory leaks were found in error paths (auth, _getsock,
flt-trace)
- only one "users" option in userlist "group" directive is supported,
but extraneous ones were still accepted and silently leaked, which
is no longer the case (an alert is now displayed when "users" is
repeated).
- FCGI would always force the status to 302 when seeing a Location
header, possibly overwriting another status code.
- http-checks could mistakenly add a "Content-Length: 0" to GET/HEAD/etc
requests, which was rejected by some servers. Now the header will only
be emitted when there is explicit content.
- H1 responses truncated after a chunk boundary (i.e. only missing the
0-sized chunk) forwarded to H2 could end up with a clean END_STREAM
flag instead of an RST_STREAM(CANCEL). The difference is subtle,
because the former states that the transfer was complete while the
latter says it was interrupted. In the first case, a client would
consider the object as complete (i.e. it could display a broken image)
while for the latter the client might possibly decide to try again.
- a few crashes could happen in the QUIC mux failed to initialize.
- some QUIC crypto frames could be 1 to 2 bytes smaller than permitted
by the MTU. Also, related to packet length, some packets can use a
long header, and some room could be missing in the buffer to store
their length field, resulting in errors.
- the signature algorithms were not listed on "show ssl crt-list". They
now are.
- a pending close from the server could be forwarded to the client
despite a pending tcp-response content evaluation.
And there are a few other extra minor ones I'm not even able to describe.
Add to this a few doc updates and cleanups, and that's roughly all.
Just like for 3.1.6, there's no rush (otherwise 3.0 wouldn't have slept
for almost two months without being released). If you're facing one of
the issues above, you can update, otherwise there's no emergency. But
please take the time to test it so that we have a solid reference in time
to qualify future reports. In the next one, I'd like to merge the fixes
that will strenghthen the FD migration between threads and make us certain
we never face reports coming from epoll races anymore. For now we don't
know if we're seeing them but they could have explained certain reports
of spurious 502. Since they're super rare, this series can wait.
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : https://www.haproxy.org/download/3.0/src/
Git repository : https://git.haproxy.org/git/haproxy-3.0.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy-3.0.git
Changelog : https://www.haproxy.org/download/3.0/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
Willy
---
Complete changelog :
Amaury Denoyelle (9):
BUG/MEDIUM: mux-quic: do not attach on already closed stream
MINOR: mux-quic: change return value of qcs_attach_sc()
BUG/MINOR: mux-quic: handle closure of uni-stream
BUG/MINOR: quic: reserve length field for long header encoding
BUG/MINOR: quic: fix CRYPTO payload size calcul for encoding
BUG/MINOR: quic: prevent crash on conn access after MUX init failure
BUG/MINOR: mux-quic: prevent crash after MUX init failure
BUG/MINOR: h3: do not report transfer as aborted on preemptive response
CLEANUP: h3: fix documentation of h3_rcv_buf()
Aurelien DARRAGON (11):
DOC: management: rename some last occurences from domain "dns" to
"resolvers"
BUG/MINOR: sink: add tempo between 2 connection attempts for sft servers
MINOR: clock: always use atomic ops for global_now_ms
CLEANUP: log: removing "log-balance" references
BUG/MINOR: log: set proper smp size for balance log-hash
BUG/MINOR: cfgparse/peers: fix inconsistent check for missing peer server
BUG/MINOR: cfgparse/peers: properly handle ignored local peer case
BUG/MINOR: server: dont return immediately from parse_server() when
skipping checks
MINOR: cfgparse/peers: provide more info when ignoring invalid "peer" or
"server" lines
BUG/MINOR: stats: fix capabilities and hide settings for some generic
metrics
BUG/MEDIUM: hlua/cli: fix cli applet UAF in hlua_applet_wakeup()
Christopher Faulet (17):
BUG/MINOR: spoe: Check the shared waiting queue to shut applets during
stopping
BUG/MINOR: spoe: Allow applet creation when closing the last one during
stopping
BUG/MEDIUM: spoe: Don't wakeup idle applets in loop during stopping
REGTESTS: Fix truncated.vtc to send 0-CRLF
BUG/MINOR: cli: Wait for the last ACK when FDs are xferred from the old
worker
BUG/MEDIUM: filters: Handle filters registered on data with no payload
callback
BUG/MINOR: fcgi: Don't set the status to 302 if it is already set
BUG/MINOR: mux-h2: Properly handle full or truncated HTX messages on shut
BUG/MINOR: tcp-rules: Don't forward close during tcp-response content
rules eval
BUG/MINOR: cli: Don't set SE flags from the cli applet
BUG/MINOR: cli: Fix memory leak on error for _getsocks command
BUG/MINOR: cli: Fix a possible infinite loop in _getsocks()
BUG/MINOR: config/userlist: Support one 'users' option for 'group'
directive
BUG/MINOR: auth: Fix a leak on error path when parsing user's groups
BUG/MINOR: flt-trace: Support only one name option
BUG/MINOR: stats-json: Define JSON_INT_MAX as a signed integer
BUG/MINOR: http-check: Don't pretend a C-L heeader is set before adding it
Damien Claisse (1):
BUG/MINOR: cfgparse-tcp: relax namespace bind check
Dragan Dosen (2):
BUG/MEDIUM: server: properly initialize PROXY v2 TLVs
BUG/MINOR: server: fix the "server-template" prefix memory leak
Ilia Shipitsin (2):
BUG/MINOR: namespace: handle a possible strdup() failure
BUG/MINOR: ssl_crtlist: handle a possible strdup() failure
Lukas Tribus (1):
DOC: option redispatch should mention persist options
Valentine Krasnobaeva (3):
MINOR: startup: adjust alert messages, when capabilities are missed
BUG/MINOR: cfgparse: fix NULL ptr dereference in cfg_parse_peers
BUG/MEIDUM: startup: return to initial cwd only after
check_config_validity()
William Lallemand (9):
BUG/MEDIUM: ssl: chosing correct certificate using RSA-PSS with TLSv1.3
BUG/MEDIUM: htx: wrong count computation in htx_xfer_blks()
DOC: htx: clarify <mark> parameter for htx_xfer_blks()
TESTS: ist: fix wrong array size
CI: github: fix h2spec.config proxy names
BUILD: ssl: allow to build without the renegotiation API of WolfSSL
BUILD: ssl: more cleaner approach to WolfSSL without renegotiation
BUG/MINOR: ssl/cli: "show ssl crt-list" lacks client-sigals
BUG/MINOR: ssl/cli: "show ssl crt-list" lacks sigals
Willy Tarreau (23):
BUG/MEDIUM: fd: mark FD transferred to another process as FD_CLONED
BUG/MINOR: stream: fix age calculation in "show sess" output
BUG/MEDIUM: thread: use pthread_self() not ha_pthread[tid] in set_affinity
BUG/MEDIUM: debug: close a possible race between thread dump and panic()
BUG/MINOR: mux-h1: always make sure h1s->sd exists in h1_dump_h1s_info()
MINOR: tinfo: add a new thread flag to indicate a call from a sig handler
BUG/MEDIUM: stream: never allocate connection addresses from signal
handler
MINOR: freq_ctr: provide non-blocking read functions
BUG/MEDIUM: stream: use non-blocking freq_ctr calls from the stream dumper
BUG/MINOR: h2: always trim leading and trailing LWS in header values
BUG/MINOR: server: check for either proxy-protocol v1 or v2 to send hedaer
BUG/MEDIUM: stream: don't use localtime in dumps from a signal handler
MINOR: compiler: add a simple macro to concatenate resolved strings
MINOR: compiler: add a new __decl_thread_var() macro to declare local
variables
MINOR: tools: resolve main() only once in resolve_sym_name()
MINOR: tools: use only opportunistic symbols resolution
BUILD: tools: silence a build warning when USE_THREAD=0
MINOR: tinfo: split the signal handler report flags into 3
MINOR: cli: export cli_io_handler() to ease symbol resolution
MINOR: tools: improve symbol resolution without dl_addr
MINOR: tools: ease the declaration of known symbols in resolve_sym_name()
MINOR: tools: teach resolve_sym_name() a few more common symbols
BUILD: tools: avoid a build warning on gcc-4.8 in resolve_sym_name()
---
