Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-27 Thread Boyang Chen
Hey all, we want to piggy-back one more protocol change to this KIP, which is the InitProducerId. Previously we overlooked the case returned from InitPid where the error code is still INVALID_PRODUCER_EPOCH instead of PRODUCER_FENCED. To be consistent, we will bump this protocol and always return

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-24 Thread Boyang Chen
Thanks a lot John for giving a review on Friday night! And thank you Guozhang, and Jason for your votes as well :) Now that we have collected 3 binding votes (Guozhang, Jason, John), I will close the voting thread and mark the KIP as approved. Still feel free to raise any question on the mailing l

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-24 Thread John Roesler
Hi Boyang, Thanks for the KIP! I've just read it over and caught up on all the prior discussions. The current version of the KIP looks good to me, and I think the decisions you've made are reasonable. I'm +1 (binding) Thanks, -John On Wed, Apr 22, 2020, at 12:12, Boyang Chen wrote: > Hey Jaso

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-22 Thread Boyang Chen
Hey Jason, thanks for the suggestions! Addressed in the KIP. On Wed, Apr 22, 2020 at 9:21 AM Jason Gustafson wrote: > +1 Just a couple small comments: > > 1. My comment about `initTransactions()` usage in the javadoc above appears > not to have been addressed. > 2. For the handling of INVALID_P

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-22 Thread Jason Gustafson
+1 Just a couple small comments: 1. My comment about `initTransactions()` usage in the javadoc above appears not to have been addressed. 2. For the handling of INVALID_PRODUCER_EPOCH in the produce response, would we only try to abort if the broker supports the newer protocol version? I guess it w

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-17 Thread Guozhang Wang
Sounds good to me. Thanks Boyang. On Fri, Apr 17, 2020 at 3:32 PM Boyang Chen wrote: > Thanks Guozhang, > > I think most of the complexity comes from our intention to benefit older > clients. After a second thought, I think the add-on complexity counteracts > the gain here as only 2.5 client is

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-17 Thread Boyang Chen
Thanks Guozhang, I think most of the complexity comes from our intention to benefit older clients. After a second thought, I think the add-on complexity counteracts the gain here as only 2.5 client is getting a slice of the resilience improvement, not for many older versions. So I decide to drop

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-17 Thread Guozhang Wang
Hi Boyang, Your reply to 3) seems conflicting with your other answers which is a bit confusing to me. Following your other answers, it seems you suggest returning UNKNOWN_PRODUCER_ID so that 2.5 clients can trigger retry logic as well? To complete my reasoning here as a complete picture: a) post

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-16 Thread Boyang Chen
Thanks Jason and Guozhang for the thoughts. On Thu, Apr 16, 2020 at 6:09 PM Guozhang Wang wrote: > For 2/3 above, originally I was not thinking that we will have a different > exception for INVALID_PRODUCER_EPOCH and hence was thinking that in order > to leverage KIP-360 for it, we'd have to let

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-16 Thread Guozhang Wang
For 2/3 above, originally I was not thinking that we will have a different exception for INVALID_PRODUCER_EPOCH and hence was thinking that in order to leverage KIP-360 for it, we'd have to let the broker to return UNKNOWN_PRODUCER_ID. I.e. we'd change the logic of partition leader as well to retur

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-16 Thread Jason Gustafson
Hi Boyang, A few minor questions below: 1. You mention UNKNOWN_PRODUCER_ID in 2.a under Resilience Improvements. I assume that should be INVALID_PRODUCER_EPOCH? I am not sure this case makes sense for 2.5 clients which would view this error as fatal regardless of whatever the broker does. Not sur

Re: [Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-16 Thread Guozhang Wang
+1 (binding), thanks! On Tue, Apr 14, 2020 at 4:36 PM Boyang Chen wrote: > Hey all, > > I would like to start the vote for KIP-588: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-588 > %3A+Allow+producers+to+recover+gracefully+from+transaction+timeouts > > Feel free to continue posting

[Vote] KIP-588: Allow producers to recover gracefully from transaction timeouts

2020-04-14 Thread Boyang Chen
Hey all, I would like to start the vote for KIP-588: https://cwiki.apache.org/confluence/display/KAFKA/KIP-588 %3A+Allow+producers+to+recover+gracefully+from+transaction+timeouts Feel free to continue posting on discussion thread if you have any questions, thanks! Best, Boyang