During the discussion of KIP-216 (https://cwiki.apache.org/confluence/display/KAFKA/KIP-216%3A+IQ+should+throw+different+exceptions+for+different+errors) we encountered that KIP-535 introduces a behavior change that is not backward compatible, hence, I would like to request a small change.
KIP-535 suggests, that active tasks can be queried during recovery and no exception would be thrown and longer. This is a change in behavior and in fact introduces a race condition for users that only want to query consistent state. Querying inconsistent state should be an opt-in, and for StandbyTasks, user can opt-in by querying them or opt-out by not querying them. However, for active task, if we don't throw an exception during recovery, users cannot opt-out of querying potentially inconsistent state, because they would need to check the state (ie, == RUNNING) before they would query the active task -- however, the state might change at any point in between, and there is a race. Hence, I would suggest to actually not change the default behavior of existing methods and we should throw an exception during restore if an active task is queried. To allow user to opt-in to query an active task during restore, we would add an overload > KafkaStream#store(..., boolean allowQueryWhileStateIsRestoring) (with an hopefully better/short variable name). Developers would use this new method to opt-into querying active tasks during restore. Thoughts? -Matthias On 11/18/19 10:45 AM, Vinoth Chandar wrote: > Thanks, everyone involved! > > On Mon, Nov 18, 2019 at 7:51 AM John Roesler <j...@confluent.io> wrote: > >> Thanks to you, also, Navinder! >> >> Looking forward to getting this feature in. >> -John >> >> On Sun, Nov 17, 2019 at 11:34 PM Navinder Brar >> <navinder_b...@yahoo.com.invalid> wrote: >>> >>> Hello all, >>> >>> With 4 binding +1 votes from Guozhang Wang, Matthias J. Sax, Bill Bejeck, >>> and John Roesler, the vote passes. >>> Thanks Guozhang, Matthias, Bill, John, Sophie for the healthy >> discussions and Vinoth for all the help on this KIP. >>> Best, >>> Navinder >>> >>> On Friday, 15 November, 2019, 11:32:31 pm IST, John Roesler < >> j...@confluent.io> wrote: >>> >>> I'm +1 (binding) as well. >>> >>> Thanks, >>> -John >>> >>> On Fri, Nov 15, 2019 at 6:20 AM Bill Bejeck <bbej...@gmail.com> wrote: >>>> >>>> +1 (binding) >>>> >>>> On Fri, Nov 15, 2019 at 1:11 AM Matthias J. Sax <matth...@confluent.io >>> >>>> wrote: >>>> >>>>> +1 (binding) >>>>> >>>>> >>>>> On 11/14/19 3:48 PM, Guozhang Wang wrote: >>>>>> +1 (binding), thanks for the KIP! >>>>>> >>>>>> Guozhang >>>>>> >>>>>> On Fri, Nov 15, 2019 at 4:38 AM Navinder Brar >>>>>> <navinder_b...@yahoo.com.invalid> wrote: >>>>>> >>>>>>> Hello all, >>>>>>> >>>>>>> I'd like to propose a vote for serving interactive queries during >>>>>>> Rebalancing, as it is a big deal for applications looking for high >>>>>>> availability. With this change, users will have control over the >>>>> tradeoff >>>>>>> between consistency and availability during serving. >>>>>>> The full KIP is provided here: >>>>>>> >>>>>>> >>>>> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-535%3A+Allow+state+stores+to+serve+stale+reads+during+rebalance >>>>>>> >>>>>>> >>>>>>> Thanks, >>>>>>> Navinder >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>> >> >
signature.asc
Description: OpenPGP digital signature
