Hi Tristan.
On 2023-11-20 (Mo.) 15:14, Tristan wrote:
Hi Aleksandar,
On 20 Nov 2023, at 17:18, Aleksandar Lazic <al-hapr...@none.at> wrote:
at configuration Change the reload leaves the old processes alive until
"hard-stop-after" value and after that is the connection terminated which does
not looks like that the connection was takeover to the new process. The use
case was log shipping with HAProxy with mode tcp, as far as I have understood
the author in a proper way.
Is that new behavior? Because I was under the impression that this is by design
Well I don't know as I have not the setup in use, I'm just the messenger and ask
if somebody have seen also such a behavior within tcp mode.
If the new process took over an existing L4 connection, it seems like it’d cause
strange behavior in quite a few cases due to configuration changes.
Well as there are the *_takover functions for http and fcgi, maybe there is also
such a function for tcp but I may have overseen it.
Either haproxy tries to reuse all old values and essentially needs to fork the
new
process for takeover (which then is equivalent to the current old process living
for a while), or it applies new values to the existing connection (assuming it’s
even possible in all cases) which is likely to just break it (removed frontend,
backend, or server; or timeouts changes, etc etc).
Seems like it’s just a design choice to me [1] and that HAProxy’s approach is
sort
of the only sane one…
Ofc that means potentially a lot of old processes, hence hard-stop-after and
max-reloads as tunables.
Now in a k8s environment I can imagine high churn in pods causing a lot of
server
changes and making this a problem, but the official ingress controllers seems to
generally mitigate it by using the runtime api when it can instead of hard
reloads,
and only using the latter in limited cases 🤔
Maybe the used the « community » ingress controller (bless their maintainer,
it’s
not a jab at it) which does rely more on hard reloads
Either way, sounds unlikely to be a fix for it?
I'm also not sure if the tcp mode have such a take over mechanism but it would
be nice for the hitless/seamless reload.
Tristan
[1]: Also a bit out of topic but I always found ~infinite duration TCP
connections
to be a very strange idea… So many things can go wrong (and will go wrong) if
you
depend on it… at least it’s never going to be as reliable as client side retries
or moving to UDP where possible…
Regards
Alex
