Hello, Does anyone know about the default being turned off for this setting? It seems like a good one to be turned on - why have replicas process something for which coordinator has already sent the timeout to client?
Thanks On Tue, Oct 12, 2021 at 11:06 AM S G <sg.online.em...@gmail.com> wrote: > Thanks Bowen. > Any idea why is cross_node_timeout commented out by default? That seems > like a good option to enable even as per the documentation: > # If disabled, replicas will assume that requests > # were forwarded to them instantly by the coordinator, which means that > # under overload conditions we will waste that much extra time processing > # already-timed-out requests. > > Also, taking an example from Oracle kind of RDBMS systems, is there a > command like the following that can be fired from an external script to > kill a long running query on each node: > > alter system kill session > > > > > On Tue, Oct 12, 2021 at 10:49 AM Bowen Song <bo...@bso.ng> wrote: > >> That will depend on whether you have cross_node_timeout enabled. However, >> I have to point out that set timeout to 15ms is perhaps not a good idea, >> the JVM GC can easily cause a lots of timeouts. >> On 12/10/2021 18:20, S G wrote: >> >> ok, when a coordinator node sends timeout to the client, does it mean all >> the replica nodes have stopped processing that specific query too? >> Or is it just the coordinator node that has stopped waiting for the >> replicas to return response? >> >> On Tue, Oct 12, 2021 at 10:12 AM Jeff Jirsa <jji...@gmail.com> wrote: >> >>> It sends an exception to the client, it doesnt sever the connection. >>> >>> >>> On Tue, Oct 12, 2021 at 10:06 AM S G <sg.online.em...@gmail.com> wrote: >>> >>>> Do the timeout values only kill the connection with the client or send >>>> error to the client? >>>> Or do they also kill the corresponding query execution happening on the >>>> Cassandra servers (co-ordinator, replicas etc) ? >>>> >>>> On Tue, Oct 12, 2021 at 10:00 AM Jeff Jirsa <jji...@gmail.com> wrote: >>>> >>>>> The read and write timeout values do this today. >>>>> >>>>> >>>>> https://github.com/apache/cassandra/blob/trunk/conf/cassandra.yaml#L920-L943 >>>>> >>>>> >>>>> On Tue, Oct 12, 2021 at 9:53 AM S G <sg.online.em...@gmail.com> wrote: >>>>> >>>>>> Hello, >>>>>> >>>>>> Is there a way to stop long running queries in Cassandra (versions >>>>>> 3.11.x or 4.x) ? >>>>>> The use-case is to have some kind of a circuit breaker based on >>>>>> query-time that has exceeded the client's SLAs. >>>>>> Example: If server response is useless to the client after 10 ms, >>>>>> then we could >>>>>> have a *query_killing_timeout* set to 15 ms (where additional 5ms >>>>>> allows for some buffer). >>>>>> And when that much time has elapsed, Cassandra will kill the query >>>>>> execution automatically. >>>>>> >>>>>> If this is not possible in Cassandra currently, any chance we can do >>>>>> it outside of Cassandra, like >>>>>> a shell script that monitors such long running queries (through users >>>>>> table etc) and kills the >>>>>> OS-thread responsible for that query (Looks unsafe though as that >>>>>> might leave the DB in an inconsistent state) ? >>>>>> >>>>>> We are trying this as a proactive measure to safeguard our clusters >>>>>> from any rogue queries fired accidentally or maliciously. >>>>>> >>>>>> Thanks ! >>>>>> >>>>>>
