Hi, HAProxy 3.1-dev10 was released on 2024/10/16. It added 119 new commits after version 3.1-dev9.
This release contains the last tricky pieces. First it includes some fixes for a few issues possibly causing crashes / frozen connections on QUIC, and another one still affecting the queue. In addition, one of the fixes I did recently hoping to fix some consideration for server maintenance state when leaving the queue was wrong and has the side effect of leaving a server in maintenance state after its address changed. Emeric and Aurélien figured it and the issue is now fixed. Now for the interesting pieces, I can try to summarize them like this: - HTTP/2: the dynamic stream window system discussed in August was finally implemented. Uploads to a 100ms server jumped from 5 Mbps to 120 Mbps without changing the default settings. The max window can be configured so as to further improve the speed. Since the buffers are dynamically allocated, they're only consumed if there's actual data so the overhead caused by large window being configured only reflects itself with slow servers. The benefit is also visible on the backend where H2 servers will no longer face a tradeoff between being slow and causing head-of-line blocking. Note that the algorithm has room for improvement, but I expect very low returns for high investments, so I think we've reached a sweet spot here. Those dealing with H2 uploads are encouraged to give it a try and to look up "tune.h2.fe.rxbuf" in the doc. The H2 traces were also completed to report more important info (connection and stream flags etc), so they're now larger but finally they're useful. - master/worker: the architecture chance discussed just before 3.0 was finally merged. It's a lot nicer and more predictable, in that the master process starts, starts the worker and waits. The master process no longer parses the worker's conf to try to undo it later. The separation between processes is cleaner, but it affects the startup sequence a little bit. We've done our best to reflect what everyone is used to check for, but for example some worker parsing errors might appear slightly later compared to master parsing error or binding errors. We figured some inconsistencies in the previous way some documented variables were handled (fixes to come later), and startup sequences and process status reporting, including in relation with vtest, that explains some random issues on the CI for MacOS. We did expect to discover some such issues anyway given the complexity of the previous model. We're aware of a tiny regression on the "reload" command issued on the master CLI, it doesn't display the startup warnings anymore. We tried a trivial fix on it and figured we were wrong since it reported the previous worker's logs. The fix is not very complicated, but needs not to be rushed, so it's expected to be finished next week. Overall that's low importance enough that it's not a showstopper. The team is super interested in feedback, especially from those reloading often, facing errors that require rollbacks, those that might be using the master-worker more without systemd and even on non-linux sytems, as well as those using the "program" sections. We did expect to see difficult issues late in the rework, but all those met were in fact low hanging fruits that could be addressed before the merge. I hope this is a good sign ;-) - QUIC: better accounting of ACKed data significantly improved the tx performance (it actually caught up with the level it had before the recent changes in dev8 or dev9). Also QUIC is now compatible with the "http-request wait-handshake" action, allowing to wait for handshake completion on selected 0-rtt requests. - ring names now support forward references (like use_backend that can reference later backends). That's important to permit log and trace statements to reference rings that are generally declared further in the config. - a do-log action was added in about all rulesets. It allows to produce logs at will using rules. This can be useful for debugging or reporting special conditions. If you're using "set-log-level silent" in your configs, you may be interested in having a look at it. - when running with wolfSSL, the server-side "ssl verify" statement no longer requires a "crl" directive. Some regtests were adjusted to compensate for some timing issues, some build warnings were addressed, and a few updates were made on the CI. We're still quite busy figuring the root cause of some of the strange pending issues affecting 3.0 (502, etc), but actually that's not that bad, because it's really time to polish what's already merged. From now on, I'd like that we don't perform massive changes anymore, only small stuff to complete what's already there, fixes, doc updates and cleanups. So this version can almost be seen as a first release candidate. Please find the usual URLs below : Site index : https://www.haproxy.org/ Documentation : https://docs.haproxy.org/ Wiki : https://github.com/haproxy/wiki/wiki Discourse : https://discourse.haproxy.org/ Slack channel : https://slack.haproxy.org/ Issue tracker : https://github.com/haproxy/haproxy/issues Sources : https://www.haproxy.org/download/3.1/src/ Git repository : https://git.haproxy.org/git/haproxy.git/ Git Web browsing : https://git.haproxy.org/?p=haproxy.git Changelog : https://www.haproxy.org/download/3.1/src/CHANGELOG Dataplane API : https://github.com/haproxytech/dataplaneapi/releases/latest Pending bugs : https://www.haproxy.org/l/pending-bugs Reviewed bugs : https://www.haproxy.org/l/reviewed-bugs Code reports : https://www.haproxy.org/l/code-reports Latest builds : https://www.haproxy.org/l/dev-packages Willy --- Complete changelog : Amaury Denoyelle (12): BUG/MAJOR: mux-quic: do not crash on empty STREAM frame emission MINOR: quic: strengthen qc_release_frm() MEDIUM: quic: decount acknowledged data for MUX txbuf window MINOR: quic: implement dedicated type for out-of-order stream ACK MEDIUM: quic: merge contiguous/overlapping buffered ack stream range MEDIUM: quic: decount out-of-order ACK data range for MUX txbuf window BUG/MINOR: quic: fix discarding of already stored out-of-order ACK BUG/MEDIUM: quic: properly decount out-of-order ACK on stream release BUG/MEDIUM: mux-quic: ensure timeout server is active for short requests MINOR: quic: notify connection layer on handshake completion BUG/MINOR: stream: unblock stream on wait-for-handshake completion BUG/MEDIUM: quic: support wait-for-handshake Aurelien DARRAGON (10): MINOR: log: add do_log() logging helper MINOR: log: add do_log_parse_act() helper func MINOR: action: add do-log action REGTESTS: add some tests for 'do-log' action BUG/MEDIUM: hlua: make hlua_ctx_renew() safe BUG/MEDIUM: hlua: properly handle sample func errors in hlua_run_sample_{fetch,conv}() MEDIUM: sink: implement sink_find_early() MINOR: trace: postresolve sink names MINOR: sample: postresolve sink names in debug() converter BUG/MEDIUM: server: server stuck in maintenance after FQDN change Christopher Faulet (1): BUG/MINOR: stats: Fix the name for the total number of streams created Damien Claisse (1): MINOR: ssl: disable server side default CRL check with WolfSSL Valentine Krasnobaeva (60): REGTESTS: cli: add delay 0.1 before connect to cli MINOR: startup: add O_CLOEXEC flag to open /dev/null MEDIUM: startup: move daemonization fork in init MINOR: startup: refactor "daemonization" fork MEDIUM: startup: move PID handling in init() MAJOR: mworker: move master-worker fork in init() BUG/MINOR: mworker: fix memory leak due to master-worker fork REORG: mworker: set nbthread=1 for master after fork MINOR: init: check MODE_MWORKER before creating master CLI REORG: mworker: move mworker_create_master_cli in master 'case' MEDIUM: startup: call chroot() if needed in one place MEDIUM: startup: do set_identity() if needed in one place MINOR: startup: only worker gets capabilities from bin CLEANUP: haproxy: rm no longer used mworker_reexec_waitmode MINOR: startup: rename exit_on_waitmode_failure to exit_on_failure MINOR: defaults: update MASTER_MAXCONN description MEDIUM: startup: remove MODE_MWORKER_WAIT MINOR: global: add MODE_DISCOVERY flag MEDIUM: cfgparse: add KWF_DISCOVERY keyword flag MEDIUM: cfgparse: call some parsers only in MODE_DISCOVERY MEDIUM: cfgparse-global: parse only KWF_DISCOVERY keywords in MODE_DISCOVERY MEDIUM: cfgparse: parse only "global" section in MODE_DISCOVERY MEDIUM: startup: introduce load_cfg and read_cfg MINOR: cfgparse: fix *thread keywords sensitive to global section position MINOR: mworker/cli: rename mworker_cli_proxy_new_listener MINOR: mworker/cli: rename and clean mworker_cli_sockpair_new MINOR: mworker/cli: create master CLI sockpair before fork MINOR: mworker/cli: create MASTER proxy before mcli listeners MINOR: mworker: add and set state PROC_O_INIT for new worker MEDIUM: mworker/cli: close child and parent fds, setup listeners MINOR: mworker: mworker_catch_sigchld: use fd_delete instead of close MINOR: startup: rename and adapt reexec_on_failure MINOR: mworker: add support for case when new worker dies MINOR: mworker: simplify the code that sets PROC_O_LEAVING MINOR: mworker/cli: add _send_status to support state transition MEDIUM: startup: split sending oldpids_sig logic for standalone and mworker modes MINOR: startup: split init() into separate initialization routines MINOR: startup: split main: add step_init_3 MINOR: startup: simplify check for calling sock_get_old_sockets MINOR: startup: encapsulate sock_get_old_sockets in a function MINOR: startup: add bind_listeners MINOR: startup: split main: add step_init_4 MINOR: startup: encapsulate master's code in run_master MINOR: startup: add read_cfg_in_discovery_mode MINOR: mworker: adapt exit_on_failure for master recovery mode MEDIUM: mworker: add support of master recovery mode MINOR: startup: add set_verbosity MEDIUM: mworker: block reloads MINOR: mworker: slow load status delivery if worker is starting MINOR: mworker: readapt program support in mworker_catch_sigchld MINOR: mworker: deserialize process list before read_cfg_in_discovery_mode MINOR: mworker: parse program only in MODE_DISCOVERY MINOR: cfgparse: add support for program section MINOR: startup: reintroduce program support MINOR: mworker-prog: stop old programs in mworker_ext_launch_all MINOR: mworker: reintroduce systemd support MINOR: mworker: report explicitly when worker exits due to max reloads MINOR: cfgparse-global: parse *env keywords in MODE_DISCOVERY MINOR: startup: reintroduce *env keywords support MINOR: startup: close devnullfd, when daemon mode is applied William Lallemand (2): MINOR: cfgparse: simulate long configuration parsing with force-cfg-parser-pause CI: cirrus-ci: bump FreeBSD image to 14-1 Willy Tarreau (33): BUILD: cache: silence an uninitialized warning at -Og with gcc-12.2 BUG/MINOR: mux-h2/traces: present the correct buffer for trailers errors traces MINOR: mux-h2/traces: print the size of the DATA frames CLEANUP: muxes: remove useless inclusion of ebmbtree.h REORG: buffers: move some of the heavy functions from buf.h to buf.c MINOR: buffer: add a buffer list type with functions MINOR: mux-h2: split the amount of rx data from the amount to ack MINOR: mux-h2: create and initialize an rx offset per stream MEDIUM: mux-h2: start to update stream when sending WU MEDIUM: mux-h2: start to introduce the window size in the offset calculation MINOR: mux-h2: count within a connection, how many streams are receiving data MINOR: mux-h2: allocate the array of shared rx bufs in the h2c MINOR: mux-h2: add rxbuf head/tail/count management for h2s MINOR: mux-h2: move H2_CF_WAIT_IN_LIST flag away from the demux flags MINOR: mux-h2: simplify the exit code in h2_rcv_buf() MINOR: mux-h2: simplify the wake up code in h2_rcv_buf() MINOR: mux-h2: clear up H2_CF_DEM_DFULL and H2_CF_DEM_SHORT_READ ambiguity MAJOR: mux-h2: make streams use the connection's buffers MAJOR: mux-h2: permit a stream to allocate as many buffers as desired MAJOR: mux-h2: make the rxbuf allocation algorithm a bit smarter MINOR: mux-h2: add tune.h2.be.rxbuf and tune.h2.fe.rxbuf global settings MEDIUM: mux-h2: change the default initial window to 16kB DOC: design-thoughts: add diagrams illustrating an rx win groth MEDIUM: mux-h2: rework h2_restart_reading() to differentiate recv and demux OPTIM: mux-h2: make h2_send() report more accurate wake up conditions OPTIM: mux-h2: try to continue reading after demuxing when useful OPTIM: mux-h2: use tasklet_wakeup_after() in h2s_notify_recv() MINOR: mux-h2/traces: add missing flags and proxy ID in traces MINOR: mux-h2/traces: add buffer-related info to h2s and h2c REGTESTS: fix a reload race in abns_socket.vtc MINOR: activity/memprofile: always return "other" bin on NULL return address BUG/MEDIUM: queue: make sure never to queue when there's no more served conns DEBUG: mux-h2/flags: add H2_CF_DEM_RXBUF & H2_SF_EXPECT_RXDATA for the decoder ---