Hi Vinoth, Thanks for the feedback. Can we link the JIRA, discussion thread also to the KIP.>> Added. Based on the discussion on KAFKA-6144, I was under the impression that this KIP is also going to cover exposing of the standby information in StreamsMetadata and thus subsume KAFKA-8994 . That would require a public API change?>> Sure, I can add changes for 8994 in this KIP and link KAFKA-6144 to KAFKA-8994 as well. KIP seems to be focussing on restoration when a new node is added. KIP-441 is underway and has some major changes proposed for this. It would be good to clarify dependencies if any. Without KIP-441, I am not very sure if we should allow reads from nodes in RESTORING state, which could amount to many minutes/few hours of stale reads? This is different from allowing querying standby replicas, which could be mostly caught up and the staleness window could be much smaller/tolerable. (once again the focus on KAFKA-8994).>> I get your point. But suppose there is a replica which has just become active, so in that case replica will still be building itself from scratch and this active will go to restoring state till it catches up with previous active, wouldn't serving from a restoring active make more sense than a replica in such case.
Finally, we may need to introduce a configuration to control this. Some users may prefer errors to stale data. Can we also add it to the KIP?>> Will add this. Regards, Navinder On2019/10/14 16:56:49, Vinoth Chandar <v...@confluent.io>wrote: >Hi Navinder,> > >Thanks for sharing the KIP! Few thoughts> > >- Can we link the JIRA, discussion thread also to the KIP> >- Based on the discussion on KAFKA-6144, I was under the impression that> >this KIP is also going to cover exposing of the standby information in> >StreamsMetadata and thus subsume KAFKA-8994 . That would require a public> >API change?> >- KIP seems to be focussing on restoration when a new node is added.> >KIP-441 is underway and has some major changes proposed for this. It would> >be good to clarify dependencies if any. Without KIP-441, I am not very sure> >if we should allow reads from nodes in RESTORING state, which could amount> >to many minutes/few hours of stale reads? This is different fromallowing> >querying standby replicas, which could be mostly caught up and the> >staleness window could be much smaller/tolerable. (once again the focus on> >KAFKA-8994)> >- Finally, we may need to introduce a configuration to control this. Some> >users may prefer errors to stale data. Can we also add it to the KIP?> > >Thanks> >Vinoth> > > > > >On Sun, Oct 13, 2019 at 3:31 PM Navinder Brar> ><na...@yahoo.com.invalid>wrote:> > >> Hi,> >> Starting a discussion on the KIP to Allow state stores to serve stale> >> reads during rebalance(> >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-535%3A+Allow+state+stores+to+serve+stale+reads+during+rebalance> >> ).> >> Thanks & Regards,Navinder> >> LinkedIn> >>> >
