> It's normal then, as it's mandated by the HTTP/2 spec to reject requests > containing any connection-specific header fields
In that case, haproxy should be consistent in it’s way of handling clients sending connection-specific headers: $ curl 'https://dashboard.domain.com/js/app.js?v=1' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:57.0) Gecko/20100101 Firefox/57.0' --compressed -H 'Connection: keep-alive' -o /dev/null -vvv % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* Trying 178.63.183.40... * TCP_NODELAY set * Connected to dashboard.domain.com (178.63.183.xxx) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH * successfully set certificate verify locations: * CAfile: /etc/ssl/cert.pem CApath: none * TLSv1.2 (OUT), TLS handshake, Client hello (1): } [512 bytes data] * TLSv1.2 (IN), TLS handshake, Server hello (2): { [93 bytes data] 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0* TLSv1.2 (IN), TLS handshake, Certificate (11): { [3000 bytes data] * TLSv1.2 (IN), TLS handshake, Server key exchange (12): { [333 bytes data] * TLSv1.2 (IN), TLS handshake, Server finished (14): { [4 bytes data] * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): } [70 bytes data] * TLSv1.2 (OUT), TLS change cipher, Client hello (1): } [1 bytes data] * TLSv1.2 (OUT), TLS handshake, Finished (20): } [16 bytes data] * TLSv1.2 (IN), TLS change cipher, Client hello (1): { [1 bytes data] * TLSv1.2 (IN), TLS handshake, Finished (20): { [16 bytes data] * SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256 * ALPN, server did not agree to a protocol * Server certificate: * subject: OU=Domain Control Validated; CN=*.domain.com * start date: Jan 3 11:17:55 2017 GMT * expire date: Jan 4 11:17:55 2018 GMT * subjectAltName: host "dashboard.domain.com" matched cert's "*.domain.com" * issuer: C=BE; O=GlobalSign nv-sa; CN=AlphaSSL CA - SHA256 - G2 * SSL certificate verify ok. > GET /js/app.js?v=1 HTTP/1.1 > Host: dashboard.domain.com > Accept: */* > Accept-Encoding: deflate, gzip > User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:57.0) > Gecko/20100101 Firefox/57.0 > Connection: keep-alive > < HTTP/1.1 200 OK < Server: nginx/1.13.5 < Date: Thu, 28 Dec 2017 10:11:34 GMT < Content-Type: application/javascript; charset=utf-8 < Last-Modified: Sun, 25 Jun 2017 17:17:05 GMT < Transfer-Encoding: chunked < Vary: Accept-Encoding < ETag: W/"594ff011-7b7" < Content-Encoding: gzip < { [695 bytes data] 100 683 0 683 0 0 3936 0 --:--:-- --:--:-- --:--:-- 3925 * Connection #0 to host dashboard.domain.com left intact Making a GET request (and sending Connection: keep-alive) continues to work, and haproxy doesn’t handle it as malformed according to the specification. So not sure if POST/PUT is handled differently within haproxy when it comes to the connection header compared to GET requests. I agree that if the specification says it cannot be done, it shouldn’t, but then at least it should be consistent. Fyi, in that case – nginx isn’t compliant with the http2 specification. I’m creating a bug report with Mozilla to see if they can change the behaviour of their browser to not send a Connection header at all, maybe this will resolve the issue. > I still have no idea what this "quantum" is by the way ;-) It’s the 2017 version of “We released a better version of Firefox that is faster”.. Let’s call it Firefox Quantum. Haproxy 2.0 could be named “haproxy Quantum 1.5” just for the giggles. Best Regards, Lucas R On 28/12/2017, 10.27, "Willy Tarreau" <[email protected]> wrote: Hi Lucas, On Thu, Dec 28, 2017 at 08:38:52AM +0000, Lucas Rolff wrote: > It worked as it should, so I started adding more and more headers, until I > hit the culprit: -H "Connection: keep-alive" or -H "Connection: close" (or > even "Connection: test") (...) It's normal then, as it's mandated by the HTTP/2 spec to reject requests containing any connection-specific header fields (Connection being the first one) : 8.1.2.2. Connection-Specific Header Fields HTTP/2 does not use the Connection header field to indicate connection-specific header fields; in this protocol, connection- specific metadata is conveyed by other means. An endpoint MUST NOT generate an HTTP/2 message containing connection-specific header fields; any message containing connection-specific header fields MUST be treated as malformed (Section 8.1.2.6). > I tried to replicate the issue in haproxy version 1.8.1, 1.8.2 and latest > commit from master - all with the same result, I also tried playing around > with the options of forceclose, http-server-close etc on both the frontend > and backend in haproxy, none of them seem to "fix" the issue. That's normal, you don't even reach this step, as it dies while decompressing the request (right after hpack decoding just before conversion from H2 to H1). > However in 1.8.2 I have 100% chance of replicating it using post requests in > Firefox and nghttp, where in 1.8.1 the issue in the majority of the time > works in Firefox and only have the few percentage failure rate. There were so many state issues with 1.8.1 that it's not much surprizing, it's possible that some of them would fail differently. > I haven't been able to replicate the issue in other than Firefox and nghttp - Clearly this means that this has to be reported to the Firefox team, as it's expected to break basically everywhere (or to help detect non-compliant servers). I still have no idea what this "quantum" is by the way ;-) > Also, sorry for the lengthy email Quite the opposite, it was extremely helpful in spotting the problem's origin. Thanks! Willy
