Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2020-07-08 Thread Rajini Sivaram
Hi all, We would like to make a change related to this KIP to ensure that transient exceptions during reassignments are handled as retriable exceptions. Kafka brokers currently return REPLICA_NOT_AVAILABLE for Fetch requests if the replica is not available on the broker. But ReplicaNotAvailableExc

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-04-08 Thread Jason Gustafson
I'm going to call this vote. +1: Me, Ryanne, Guozhang, Eno, David, Viktor, Stephane, Gwen, Jun Binding total is +5 with no -1 or +0. Thanks all for the discussion and feedback. -Jason On Thu, Apr 4, 2019 at 4:55 PM Jun Rao wrote: > Hi, Jason, > > Thanks for the updated KIP. +1 from me. > > J

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-04-04 Thread Jun Rao
Hi, Jason, Thanks for the updated KIP. +1 from me. Jun On Thu, Apr 4, 2019 at 2:26 PM Jason Gustafson wrote: > Hi Jun, > > I have updated the KIP to remove `replica.selection.policy` from the > consumer configuration. Thanks for the suggestion. > > Best, > Jason > > On Wed, Mar 27, 2019 at 9:4

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-04-04 Thread Gwen Shapira
+1 On Thu, Apr 4, 2019 at 2:26 PM Jason Gustafson wrote: > Hi Jun, > > I have updated the KIP to remove `replica.selection.policy` from the > consumer configuration. Thanks for the suggestion. > > Best, > Jason > > On Wed, Mar 27, 2019 at 9:46 AM Jason Gustafson > wrote: > > > @Jun > > > > Re;

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-04-04 Thread Jason Gustafson
Hi Jun, I have updated the KIP to remove `replica.selection.policy` from the consumer configuration. Thanks for the suggestion. Best, Jason On Wed, Mar 27, 2019 at 9:46 AM Jason Gustafson wrote: > @Jun > > Re; 200: It's a fair point that it is useful to minimize the client > changes that are n

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-27 Thread Jason Gustafson
@Jun Re; 200: It's a fair point that it is useful to minimize the client changes that are needed to get a benefit from affinity. I think the high level argument that this is mostly the concern of operators and should be under their control. Since there is a protocol bump here, users will have to u

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-27 Thread Jun Rao
Hi, Guozhang, In general, users may want to optimize affinity in different ways, e.g. latency, cost, etc. I am not sure if all those cases can by captured by client IP addresses. So, it seems that having a rack.id in the consumer is still potentially useful. Thanks, Jun On Wed, Mar 27, 2019 at

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-27 Thread Guozhang Wang
Hello Jun, Regarding 200: if we assume that most client would not bother setting rack.id at all and affinity can be determined w/o rack.id via TCP header, plus rack.id may not be "future-proof" additional information is needed as well, then do we still need to change the protocol of metadata reque

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-26 Thread Jun Rao
Hi, Jason, Thanks for the KIP. Just a couple of more comments. 200. I am wondering if we really need the replica.selection.policy config in the consumer. A slight variant is that we (1) let the consumer always fetch from the PreferredReplica and (2) provide a default implementation of ReplicaSele

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-26 Thread Stephane Maarek
It's going to change quite a few things for learners, but this is an awesome idea! +1 (non-binding) On Tue, Mar 26, 2019 at 3:35 PM Viktor Somogyi-Vass wrote: > +1 (non-binding) very good proposal. > > On Mon, Mar 25, 2019 at 7:19 PM David Arthur wrote: > > > +1 > > > > Thanks, Jason! > > > > O

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-26 Thread Viktor Somogyi-Vass
+1 (non-binding) very good proposal. On Mon, Mar 25, 2019 at 7:19 PM David Arthur wrote: > +1 > > Thanks, Jason! > > On Mon, Mar 25, 2019 at 1:23 PM Eno Thereska > wrote: > > > +1 (non-binding) > > Thanks for updating the KIP and addressing my previous comments. > > > > Eno > > > > On Mon, Mar

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-25 Thread David Arthur
+1 Thanks, Jason! On Mon, Mar 25, 2019 at 1:23 PM Eno Thereska wrote: > +1 (non-binding) > Thanks for updating the KIP and addressing my previous comments. > > Eno > > On Mon, Mar 25, 2019 at 4:35 PM Ryanne Dolan > wrote: > > > +1 (non-binding) > > > > Great stuff, thanks. > > > > Ryanne > > >

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-25 Thread Eno Thereska
+1 (non-binding) Thanks for updating the KIP and addressing my previous comments. Eno On Mon, Mar 25, 2019 at 4:35 PM Ryanne Dolan wrote: > +1 (non-binding) > > Great stuff, thanks. > > Ryanne > > On Mon, Mar 25, 2019, 11:08 AM Jason Gustafson wrote: > > > Hi All, discussion on the KIP seems t

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-25 Thread Guozhang Wang
+1 (binding). A very well written proposal and a pleasant read. Thanks Jason! Guozhang On Mon, Mar 25, 2019 at 9:07 AM Jason Gustafson wrote: > Hi All, discussion on the KIP seems to have died down, so I'd like to go > ahead and start a vote. Here is a link to the KIP: > > https://cwiki.apache

Re: [VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-25 Thread Ryanne Dolan
+1 (non-binding) Great stuff, thanks. Ryanne On Mon, Mar 25, 2019, 11:08 AM Jason Gustafson wrote: > Hi All, discussion on the KIP seems to have died down, so I'd like to go > ahead and start a vote. Here is a link to the KIP: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-392%3A+Al

[VOTE] KIP-392: Allow consumers to fetch from the closest replica

2019-03-25 Thread Jason Gustafson
Hi All, discussion on the KIP seems to have died down, so I'd like to go ahead and start a vote. Here is a link to the KIP: https://cwiki.apache.org/confluence/display/KAFKA/KIP-392%3A+Allow+consumers+to+fetch+from+closest+replica . +1 from me (duh) -Jason