Hi Pavel, Thanks for the links. Yes, I forgot that the flag of changed topology is lazy. Also I missed that the keepAlive setting is configured on the client side (alternatively to idleTimeout that is on the server side).
Now I understand, this feature can be helpful then. Every client can configure itself in case it's possible to be idle sometimes, and choose an appropriate timeout by itself too. And by default the feature should be disabled. WDYT, should we add a WARN message for clients that configure keepAliveTimeout greater than idleTimeout on the server side? On Mon, Feb 7, 2022 at 1:05 PM Pavel Tupitsyn <ptupit...@apache.org> wrote: > Ivan, > > I suggest the following: > > 1. Server sends KEEP_ALIVE feature flag, which means it accepts > OP_KEEP_ALIVE empty message > 2. Client sends OP_KEEP_ALIVE when the connection is idle for a > certain period of time > 3. Already implemented: when ClientConnectorConfiguration#idleTimeout is > not zero, server disconnects idle clients > > This way we don't need server->client keepalives, as you correctly noted. > > On Mon, Feb 7, 2022 at 12:43 PM Ivan Daschinsky <ivanda...@gmail.com> > wrote: > > > Pavel, I suppose that ideally: > > 1. Client send in handshake flag, that it supports KEEP_ALIVE feature and > > server takes it into account. > > 2. Each request of client can be considered as keep-alive ping. > > 3. Client send failure should be processed using retry policy. > > 4. Server should not send keep-alive packets, it is redundant, but server > > should track requests from client and if there is no requests from client > > with KEEP_ALIVE feature, > > automatically close connection and free resources. > > > > Similar approach is used in zookeeper clients. > > > > пн, 7 февр. 2022 г. в 12:24, Pavel Tupitsyn <ptupit...@apache.org>: > > > > > Ivan, > > > > > > Ideally, the check should come from both sides. > > > - Client periodically sends keepalive to server > > > - Server periodically sends keepalive to client > > > > > > Feature flags will be added accordingly, so it is not necessary to > > > implement this in all thin clients. > > > > > > On Mon, Feb 7, 2022 at 11:43 AM Ivan Daschinsky <ivanda...@gmail.com> > > > wrote: > > > > > > > I suppose it is great idea, but this functionality can be hard to > > > implement > > > > for some platforms. I.e. sync python client or php (there is no real > > > > multithreading for python (GIL) and php is single threaded by > design). > > > But > > > > for async clients it is not very hard to implement. Nevertheless, > this > > > > feature should be optional, because of possible technical > limitations. > > > > > > > > Pavel, is this check mostly for client side? Or servers can do some > > > actions > > > > if there is no activity from thin client (i.e. closing context and > free > > > > resources such as queries' handles and so on?) > > > > > > > > пн, 7 февр. 2022 г. в 11:09, Pavel Tupitsyn <ptupit...@apache.org>: > > > > > > > > > Hi Maksim, > > > > > > > > > > > > > > > > half-state is a possible situation when an Ignite node goes down > or > > > > > somehow removes connection to a thin client > > > > > > > > > > Half-open state is also possible when, for example, an intermediate > > > > router > > > > > is rebooted [1]. > > > > > > > > > > This is what we seem to have encountered with one of our customers > - > > > they > > > > > have a stable cluster, and long-living (multiple days) thin client > > > > > connections which can be idle for some time. > > > > > And only when we send some data on such an idle connection do we > > > discover > > > > > that it is broken. > > > > > > > > > > > > > > > > But with enabled (true by default) partitionAwareness feature > > clients > > > > can > > > > > be notified about topology changes > > > > > > > > > > Partition awareness is a "lazy" notification in a form of a > response > > > > > message flag [2]. > > > > > You won't get one on an idle connection. > > > > > > > > > > > > > > > > the connections are removed on the server side by client idle > > timeout > > > > > > > > > > Idle timeout is disabled by default. > > > > > > > > > > > > > > > > is it OK to keep such connections alive for a long time > > > > > > > > > > I think it is up to the user. > > > > > > > > > > > > > > > > in the case of partition awareness features it can lead to > wasting > > > TCP > > > > > sockets on Ignite nodes, can't it > > > > > > > > > > Can you please elaborate? > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > https://blog.stephencleary.com/2009/05/detection-of-half-open-dropped.html > > > > > [2] > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+Thin+Clients > > > > > > > > > > On Fri, Feb 4, 2022 at 4:01 PM Maksim Timonin < > > timoninma...@apache.org > > > > > > > > > wrote: > > > > > > > > > > > Hi Pavel, > > > > > > > > > > > > Thanks for starting this thread! Can I ask some questions here to > > get > > > > the > > > > > > feature more clearly? > > > > > > > > > > > > As I understand it correctly, half-state is a possible situation > > when > > > > an > > > > > > Ignite node goes down or somehow removes connection to a thin > > client. > > > > But > > > > > > with enabled (true by default) partitionAwareness feature clients > > can > > > > be > > > > > > notified about topology changes. So, there are possible cases: > > > > > > 1. ThinClient connects to a single node. > > > > > > 2. Ignite node removes connection from itself. > > > > > > > > > > > > I like the idea for the case with a single node, as it helps fail > > > fast. > > > > > > But is it OK to connect a client to a single node only? > > > > > > > > > > > > For the second one: you mention that a case for the second option > > is > > > > > > "Long-living and mostly idle connections are especially > susceptible > > > to > > > > > this > > > > > > behavior". If I understand correctly the connections are removed > on > > > the > > > > > > server side by client idle timeout. Can we just configure the > idle > > > > > timeout > > > > > > for cases where we really need keeping alive idle connections? > Are > > > > there > > > > > > any other cases with unexpectedly dropped connections? > > > > > > > > > > > > I'm wondering is it OK to keep such connections alive for a long > > > time? > > > > > > Also in the case of partition awareness features it can lead to > > > wasting > > > > > TCP > > > > > > sockets on Ignite nodes, can't it? > > > > > > > > > > > > Thanks! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Feb 3, 2022 at 2:24 PM Pavel Tupitsyn < > > ptupit...@apache.org> > > > > > > wrote: > > > > > > > > > > > >> Igniters, > > > > > >> > > > > > >> Please review the proposal to add heartbeat messages to the thin > > > > client > > > > > >> protocol (both 2.x and 3.x) and let me know your thoughts: > > > > > >> > > > > > >> > > > > > >> > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-83+Thin+Client+Keepalive > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Sincerely yours, Ivan Daschinskiy > > > > > > > > > > > > > -- > > Sincerely yours, Ivan Daschinskiy > > >
