Public bug reported:
Afterhaproxy on Ubuntu 22.04 (jammy) got updated, HAProxy started
returning 502 Bad Gateway for backend responses that use chunked transfer
encoding
and are larger than a single read. Small responses and responses with
Content-Length
are unaffected.
Package versions
- Before (working): 2.4.30-0ubuntu0.22.04.2
- After (broken): 2.4.30-0ubuntu0.22.04.3 (HAProxy version
2.4.30-0ubuntu0.22.04.3 2026/10/27)
- Upgrade time (dpkg.log): 2026-11-06 06:42:04
- First "invalid response" events: 2026-11-06 08:44, then continuously on every
backend in the same frontend. None before the upgrade.
Setup
- Ubuntu 22.04, HAProxy in mode http, TLS terminated on the frontend
(bind ... ssl crt ... alpn h2,http/1.1); the frontend is also reachable over
plain HTTP.
- Backends are nginx servers reached over plain HTTP/1.1 (some over TLS with
"ssl alpn http/1.1 verify none"). The issue affects both.
- Relevant defaults:
mode http
option http-server-close
option http-pretend-keepalive
option httpclose
option forwardfor
timeout connect 20s
timeout client 20m
timeout server 20s
- No tune.bufsize/maxrewrite overrides, no compression filter, no Lua/SPOE.
Symptom
For a ~30 KB gzip-encoded static file served chunked by the backend:
- Through HAProxy, over HTTP/1.1 or HTTP/2, from any client: 502/107 every time
("The server returned an invalid or incomplete response.")
- Same request sent with curl directly from the HAProxy host to the backend:
200, 30294 bytes, complete
- Small chunked responses (<1 KB) and Content-Length responses pass through
fine.
I captured the raw backend response and checked it. The chunked framing is valid
(4 chunks: a 10-byte first chunk holding the gzip header, then data chunks,
then the terminating 0-chunk). HAProxy accepts the 10-byte first chunk and then
fails when parsing the next chunk-size line.
"show errors" (excerpts; addresses removed)
[06/Nov/2026:14:22:30.911] backend be_http (#10): invalid response
frontend fe_main (#4), server nginx-b (#1), event #964
buffer starts at 0 (including 0 out), 16384 free,
len 0, wraps at 0, error at position 0
H1 connection flags 0x80002200, H1 stream flags 0x00004840
H1 msg state MSG_CHUNK_SIZE(26), H1 msg flags 0x00001756
H1 chunk len 0 bytes, H1 body len 10 bytes :
[06/Nov/2026:14:23:29.726] backend be_a (#8): invalid response
frontend fe_main (#4), server nginx-a (#1), event #965
buffer starts at 0 (including 0 out), 16384 free,
len 0, wraps at 0, error at position 0
H1 connection flags 0x80002200, H1 stream flags 0x00004840
H1 msg state MSG_CHUNK_SIZE(26), H1 msg flags 0x00001756
H1 chunk len 0 bytes, H1 body len 0 bytes :
[06/Nov/2026:14:07:30.069] backend be_a_tls (#9): invalid response
frontend fe_main (#4), server nginx-a (#1), event #912
buffer starts at 0 (including 0 out), 16384 free,
len 0, wraps at 0, error at position 0
H1 connection flags 0x80002200, H1 stream flags 0x00004840
H1 msg state MSG_CHUNK_SIZE(26), H1 msg flags 0x00001756
H1 chunk len 0 bytes, H1 body len 47484922 bytes :
[06/Nov/2026:14:20:41.401] backend be_b_20m (#13): invalid response
frontend fe_main (#4), server nginx-b (#1), event #952
buffer starts at 0 (including 0 out), 16384 free,
len 0, wraps at 0, error at position 0
H1 connection flags 0x80002200, H1 stream flags 0x00004840
H1 msg state MSG_CHUNK_SIZE(26), H1 msg flags 0x00001756
H1 chunk len 0 bytes, H1 body len 997 bytes :
The same state also shows up on the request side, for chunked uploads
from clients:
[06/Nov/2026:14:18:26.356] frontend fe_main (#4): invalid request
backend be_b_20m (#13), server <NONE> (#-1), event #947
buffer starts at 0 (including 0 out), 16384 free,
len 0, wraps at 0, error at position 0
H1 connection flags 0x00002200, H1 stream flags 0x00000810
H1 msg state MSG_CHUNK_SIZE(26), H1 msg flags 0x00001752
H1 chunk len 0 bytes, H1 body len 10739 bytes :
The counter went up by ~350 events in about 4 hours across 6 backends, all with
MSG_CHUNK_SIZE / "len 0" / "error at position 0".
Other causes checked and ruled out
- Backend output: the same requests sent directly to the backend (from the
HAProxy
host and from other hosts) return complete, correctly framed chunked
responses.
- Network path: a direct request from the HAProxy host to the backend gets the
full
body. tcpdump of HAProxy <-> backend shows no FIN/RST from the backend before
HAProxy gives up.
- Backend client-abort handling: nginx "proxy_ignore_client_abort on" changes
nothing.
Steps to reproduce
1. Ubuntu 22.04 with haproxy 2.4.30-0ubuntu0.22.04.3 and the defaults above,
frontend in mode http, backend = nginx proxying an application that returns
gzip-encoded chunked responses (here a Jenkins static asset, ~30 KB gzipped).
2. curl -s -o /dev/null --compressed -w '%{http_code}/%{size_download}\n'
https://<frontend>/<large-chunked-resource>
Expected: 200/30294 Actual: 502/107
3. echo "show errors" | socat stdio <stats socket> -> MSG_CHUNK_SIZE invalid
response
Workaround
DOWNGRADE TO 2.4.30-0ubuntu0.22.04.2 + apt-mark hold
Expected behaviour
Valid chunked responses are forwarded unchanged, as in 2.4.30-0ubuntu0.22.04.2.
** Affects: haproxy (Ubuntu)
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/2173557
Title:
haproxy 2.4.30-0ubuntu0.22.04.3 regression: valid chunked responses
rejected after first chunk ("invalid response", MSG_CHUNK_SIZE) → 502
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/haproxy/+bug/2173557/+subscriptions
--
ubuntu-bugs mailing list
[email protected]
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs
