On Tue, Oct 24, 2023 at 10:35 AM Willy Tarreau <w...@1wt.eu> wrote: > > We are running 2.9-dev8 for the server connection close fix for > > "not-so-great" gRPC clients. We just experienced an ha_panic seemingly > > triggered from OpenSSL 3. This is a fairly default Ubuntu 22.04 > > system, with locally built HAProxy package (as there are no "official" > > dev builds). > > Hmm that's not cool. Did it happen only once or repeatedly ?
We are running multiple load balancers, and it was just one host that experienced this strange issue once. I wanted to bring it up here in the list, as it happened right after we deployed 2.9-dev8, but we have not seen the same issue again since that one time. So it was most likely some strange OpenSSL 3 edge case as you said. > It's very possible, indeed. Do you have SSL on the frontend only or also > on the backend ? SSL is not used on the backend for this scenario. > Also, is you machine heavily loaded or not ? I'm trying to estimate if > it's worth switching to alternate locks in your case. In case you're > interested in giving it a try, you can add USE_PTHREAD_EMULATION=1 to > your "make" command line. It may seem to use more CPU but will in fact > replace the sleeping wait by an active wait and for a shorter time, > resulting in faster processing and a real (not just apparent) load > reporting. The system has some other services running on it (not only load balancing) so it is not sitting idle and sees periodic load, but it's a powerful box and the load experienced shouldn't be a concern in my eyes. I went ahead and rebuilt with `USE_PTHREAD_EMULATION=1`. We run Ubuntu 22.04 across our fleet and plan to keep it that way, even with the OpenSSL 3 drama. The rebuilt binary has behaved stable -- thank you for bringing attention to the flag. Overall, feels strange, but doesn't seem like there is anything actionable here in the end. Thank you in any case! Valters Jansons
