Hi,
HAProxy 2.9-dev7 was released on 2023/10/06. It added 75 new commits
after version 2.9-dev6.
This version fixes a number of issues in previous development releases
and prepares the work for subsequent patch series:
- Rémi found a bug in the latest update of the plock code that could
sometimes make the wrlock unreliable (it didn't wait for one reader
to finish). Let's hope this was the one causing the two recent
crashes on haproxy.org for which the cores made no sense but could
be explained by memory corruption.
- some updates to the reverse-connect code, which now works much better.
An FD leak was fixed, the attach-srv action parser was fixed to support
conditions and doesn't mandate anymore that all listeners are H2. I'll
retry to setup a gateway soon.
- a race in the Lua co-socket connect code was addressed, by which if
it's interrupted by the Lua scheduler in the middle of the operation,
it would fail and not be able to recover. Now it will be able to
reconnect.
- QUIC now builds with the AWS-LC TLS library. A few features are
currently missing: 0-RTT, some ciphers based on chacha20 and
aes128-ccm, and the client hello callback that's used to select
the certificate between RSA and ECDSA. The support status page
will be updated soon with this.
- QUIC binding errors at runtime when trying to connect back to a
client will now emit a warning once to suggest using capabilities
or uid to address the issue.
- the cache now supports the "Origin" header as a secondary cache key
- "http-request set-timeout" now supports "client" to adjust the timeout
on the front connection.
- the panic message for spinning streams will now use the same large
format as used in "show sess all", because once to twice a year we
still face such a report and every time the contents were insufficient
to diagnose the reason.
- the post-parsing checks for the "mode" keyword were all revisited not
to consider anymore that TCP and HTTP were mutual opposites. This will
make it easier to bring new modes.
Note that we're currently chasing a few issues:
- we managed to trigger some transfer timeouts with QUIC over lossy
networks, which depend on the object size, congestion control
algorithms, stream counts and buffer sizes. We captured a full trace
and spent a few hours analyzing it to try to estimate where the
problem was, but for now without particular conclusion except "that's
difficult". This may or may not match what Tristan saw in
listener-socket mode and what Luke also reported recently.
- on haproxy.org after some time (~10 days) we start to see a memory
usage increase related to a negative "needed_avg" estimate that
dissuades the callers from releasing the memory. I know it appeared
with the split of the pools heads in dev3, I thought I fixed it but
it had not been enough in dev6, so I need to continue to analyze the
situation (for now it does not happen). I think I managed to ease its
reproduction by using violent H2 injections, so I'll have to see. If
you observe any abnormal memory, just have a look at the output of
"show pools" and see if you see very large "needed_avg". If so you
may need to reload.
For next steps, some progress was made on long-time issue #401 (improved
log server management) with a first step consisting in finally being able
to check log servers and group them so that we won't need anymore to rely
on the sampling mechanism to perform some dirty load balancing. It might
even partially address #1519 (stats). At first glance we already have
something usable so it should be merged early next week once the review
is complete. I'm still unhappy with the TCP logs that do not scale with
threads due to the same reason as the traces: the ring lock that I already
tried to remove but that first requires that the DNS code doesn't share
this code anymore and that the buffer functions are split and taken away
from there. A second big part is the mux-to-mux forwarding that's a bit
sensitive and that allows one side to consider the other side's window and
flow control before starting to receive data. The goal here is to avoid as
much as possible to store data into buffers that we won't be able to
evacuate on the other side, since it consumes memory, thrashes L3 caches
and degrades performance. It's an even older wish that should pave the way
to new classes of optimization and that tends to confirm that the new
architecture is much more flexible than the pre-1.9 one (yes, 10 versions
will have been necessary for this). A third part is the cache's locking
which is about to be significantly relaxed. This should also be ready for
next dev.
And of course I still have to review Alexander's patches. Overall I'm glad
that the sensitive stuff is getting merged ~1.5 months before the release,
as it will let us finish the small stuff while continuing the testing and
bug fixing, so everything is still on track for now.
Please find the usual URLs below :
Site index : https://www.haproxy.org/
Documentation : https://docs.haproxy.org/
Wiki : https://github.com/haproxy/wiki/wiki
Discourse : https://discourse.haproxy.org/
Slack channel : https://slack.haproxy.org/
Issue tracker : https://github.com/haproxy/haproxy/issues
Sources : https://www.haproxy.org/download/2.9/src/
Git repository : https://git.haproxy.org/git/haproxy.git/
Git Web browsing : https://git.haproxy.org/?p=haproxy.git
Changelog : https://www.haproxy.org/download/2.9/src/CHANGELOG
Dataplane API :
https://github.com/haproxytech/dataplaneapi/releases/latest
Pending bugs : https://www.haproxy.org/l/pending-bugs
Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs
Code reports : https://www.haproxy.org/l/code-reports
Latest builds : https://www.haproxy.org/l/dev-packages
Willy
---
Complete changelog :
Amaury Denoyelle (12):
BUG/MINOR: mux-quic: remove full demux flag on ncbuf release
BUG/MINOR: proto_reverse_connect: fix FD leak on connection error
BUG/MINOR: tcp_act: fix attach-srv rule ACL parsing
MINOR: connection: define error for reverse connect
MINOR: connection: define mux flag for reverse support
MINOR: tcp_act: remove limitation on protocol for attach-srv
DEV: sslkeylogger: handle file opening error
MINOR: quic: define quic-socket bind setting
MINOR: quic: handle perm error on bind during runtime
MINOR: backend: refactor specific source address allocation
MINOR: proto_reverse_connect: support source address setting
BUG/MINOR: hq-interop: simplify parser requirement
Aurelien DARRAGON (20):
MINOR: tcp_rules: tcp-{request,response} requires TCP or HTTP mode
MINOR: stktable: "stick" requires TCP or HTTP mode
MINOR: filter: "filter" requires TCP or HTTP mode
MINOR: backend/balance: "balance" requires TCP or HTTP mode
MINOR: flt_http_comp: "compression" requires TCP or HTTP mode
MINOR: http_htx/errors: prevent the use of some keywords when not in
tcp/http mode
MINOR: fcgi-app: "use-fcgi-app" requires TCP or HTTP mode
MINOR: cfgparse-listen: "http-send-name-header" requires TCP or HTTP mode
MINOR: cfgparse-listen: "dynamic-cookie-key" requires TCP or HTTP mode
MINOR: proxy: dynamic-cookie CLIs require TCP or HTTP mode
MINOR: cfgparse-listen: "http-reuse" requires TCP or HTTP mode
MINOR: proxy: report a warning for max_ka_queue in
proxy_cfg_ensure_no_http()
MINOR: cfgparse-listen: warn when use-server rules is used in wrong mode
DOC: config: unify "log" directive doc
MINOR: sink/log: fix some typos around postparsing logic
MINOR: sink: remove useless check after sink creation
MINOR: sink: don't rely on p->parent in sink appctx
MINOR: sink: don't rely on forward_px to init sink forwarding
MINOR: sink: refine forward_px usage
MINOR: sink: function to add new sink servers
Christopher Faulet (19):
BUILD: pool: Fix GCC error about potential null pointer dereference
MINOR: hlua: Set context's appctx when the lua socket is created
MINOR: hlua: Don't preform operations on a not connected socket
MINOR: hlua: Save the lua socket's timeout in its context
MINOR: hlua: Save the lua socket's server in its context
MINOR: hlua: Test the hlua struct first when the lua socket is connecting
BUG/MEDIUM: hlua: Initialize appctx used by a lua socket on connect only
DEBUG: mux-h1: Fix event label from trace messages about payload
formatting
BUG/MINOR: mux-h1: Handle read0 in rcv_pipe() only when data receipt was
tried
BUG/MINOR: mux-h1: Ignore C-L when sending H1 messages if T-E is also set
BUG/MEDIUM: h1: Ignore C-L value in the H1 parser if T-E is also set
REGTESTS: filters: Don't set C-L header in the successful response to
CONNECT
MINOR: mux-h1: Add flags if outgoing msg contains a header about its
payload
MINOR: mux-h1: Rely on H1S_F_HAVE_CHNK to add T-E in outgoing messages
BUG/MEDIUM: mux-h1: Add C-L header in outgoing message if it was removed
BUG/MEDIUM: mux-h1; Ignore headers modifications about payload
representation
BUG/MINOR: h1-htx: Keep flags about C-L/T-E during HEAD response parsing
MINOR: h1-htx: Declare successful tunnel establishment as bodyless
BUG/MEDIUM: stconn: Fix comparison sign in sc_need_room()
Emeric Brun (1):
Revert "BUG/MEDIUM: quic: missing check of dcid for init pkt including a
token"
Remi Tricot-Le Breton (3):
Revert "MEDIUM: sample: Small fix in function check_operator for eror
reporting"
DOC: sample: Add a comment in 'check_operator' to explain why
'vars_check_arg' should ignore the 'err' buffer
MEDIUM: cache: Add "Origin" header to secondary cache key
Vladimir Vdovin (1):
MINOR: support for http-request set-timeout client
William Lallemand (3):
BUG/MINOR: proto_reverse_connect: fix FD leak upon connect
BUILD: quic: allow USE_QUIC to work with AWSLC
CI: github: add USE_QUIC=1 to aws-lc build
Willy Tarreau (16):
CLEANUP: freq_ctr: make all freq_ctr readers take a const
CLEANUP: stream: make the dump code not depend on the CLI appctx
MINOR: stream: split stats_dump_full_strm_to_buffer() in two
CLEANUP: stream: use const filters in the dump function
CLEANUP: stream: make strm_dump_to_buffer() take a const stream
MINOR: stream: make strm_dump_to_buffer() take an arbitrary buffer
MINOR: stream: make strm_dump_to_buffer() show the list of filters
MINOR: stream: make stream_dump() always multi-line
MINOR: streams: add support for line prefixes to strm_dump_to_buffer()
MEDIUM: stream: now provide full stream dumps in case of loops
MINOR: debug: use the more detailed stream dump in panics
CLEANUP: stream: remove the now unused stream_dump() function
MINOR: stream: fix output alignment of stuck thread dumps
BUG/MAJOR: plock: fix major bug in pl_take_w() introduced with EBO
MINOR: haproxy: permit to register features during boot
BUG/MEDIUM: actions: always apply a longest match on prefix lookup
---
