On Tue, Nov 14, 2023 at 05:18:14PM -0800, Abhijeet Rastogi wrote:
> Hi,
>
> I'll appreciate a response on this, I was unable to get any guidance from
> docs.
>
> Thanks
>
> On Tue, Nov 7, 2023 at 3:37 PM Abhijeet Rastogi <abhijeet.1...@gmail.com>
> wrote:
>
> > Hi HAproxy community,
> >
> > In our HAproxy deployment, access to runtime API is extremely critical, it
> > is used for:-
> > - collecting metrics
> > - managing servers in a backend (SoT is not DNS, so we have to use runtime
> > API), etc
> >
> > I observed that, when soft-stop is in progress, we get the following error
> > while performing a connect.
> >
> > connect(5, {sa_family=AF_UNIX, sun_path="path/to/haproxy.sock"}, 60) = -1
> > ECONNREFUSED (Connection refused)
> > recvfrom(3, 0x7ffcc675fe10, 519, MSG_DONTWAIT, NULL, NULL) = -1 EAGAIN
> > (Resource temporarily unavailable)
> >
> > Due to this, we're unable to perform any operations or collect metrics
> > while haproxy is in soft stop, for us, that's a long time as we use the
> > current settings.
> >
> > close-spread-time 10m
> > hard-stop-after 11m
> >
> > "stats socket" (unix socket) is configured on one of the production
> > frontends, we're using version "version 2.8.0-01b97d-15 2023/09/26".
> >
> > Will it make sense if HAproxy had an option to not disable runtime API
> > during soft-stop, perhaps an argument
> > While exploring the code, I noticed that the soft-stop code stops all
> > "stoppable
> > listeners"
> > <https://sourcegraph.com/github.com/haproxy/haproxy/-/blob/src/proxy.c?L2280>,
> > I wonder if there's a way to make a listener non-stoppable?
> >
> > Are there any workarounds to getting around this limitation, while also
> > retaining the soft-stop behavior?
> > Our reason to use soft-stop is to do best-effort into gracefully draining
> > client connections (send Connection: close). We also don't have a way to
> > drain H2 connections proactively, but that's a discussion for another day.
> >
> > Cheers,
> > Abhijeet (https://abhi.host)
> >
>
Hi Abhijeet,
A soft stop would close the stats socket of the leaving process, so if
you are not attached anymore to the socket you can't do much. If you
keep a connection to the socket you won't have the problem, but you need
to configure the right timeout so the connection won't be closed.
A cleaner solution is to use the master CLI instead of the stats
socket, this is a socket which is attached to the master process, and
which is able to connect to leaving processes.
http://docs.haproxy.org/2.8/management.html#9.4
Regards,
--
William Lallemand
