Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-09-23 Thread Matthias J. Sax
Sorry for the late reply Jason, and thanks for calling it out. After couple if direct discussion with Jason, Colin, Ismael, and John, I agree that we should keep `retries` config for both producer and admin client. I updated the KIP accordingly and opened a PR to revert the deprecation: https://g

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-07-27 Thread Jason Gustafson
Hi Matthias, Sorry for jumping in so late here. I am trying to understand why it was necessary to deprecate `retries` in the producer. One of the use cases that I see in practice is setting `retries` to 0. This allows applications to control the retry semantics themselves. For example, I have seen

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-07-20 Thread Matthias J. Sax
While working on the PR, we realized that the command line tool bin/kafka-console-producer.sh has a flag `--message-send-max-retries` to set the producer's `retries` config. We also need to deprecate this flag. I updated the KIP accordingly. Please let us know if there are any concerns to this

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-06-10 Thread Matthias J. Sax
Thanks! +1 (binding) from myself. I am closing the vote as accepted with 3 binding and 3 non-binding votes. binding: - John - Guozhang - Matthias non-binding: - Sophie - Boyang - Bruno -Matthias On 6/4/20 5:26 PM, Matthias J. Sax wrote: > Guozhang, > > what you propose makes sense,

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-06-09 Thread Guozhang Wang
Sounds fair to me. I'm +1 on the KIP. On Thu, Jun 4, 2020 at 5:26 PM Matthias J. Sax wrote: > Guozhang, > > what you propose makes sense, but this seems to get into implementation > detail territory already? > > Thus, if there are nor further change requests to the KIP wiki page > itself, I woul

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-06-04 Thread Matthias J. Sax
Guozhang, what you propose makes sense, but this seems to get into implementation detail territory already? Thus, if there are nor further change requests to the KIP wiki page itself, I would like to proceed with the VOTE. -Matthias On 5/20/20 12:30 PM, Guozhang Wang wrote: > Thanks Matthias,

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-20 Thread Guozhang Wang
Thanks Matthias, I agree with you on all the bullet points above. Regarding the admin-client outer-loop retries inside partition assignor, I think we should treat error codes differently from those two blocking calls: Describe-topic: * unknown-topic (3): add this topic to the to-be-created topic

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-18 Thread Matthias J. Sax
No worries Guozhang, any feedback is always very welcome! My reply is going to be a little longer... Sorry. > 1) There are some inconsistent statements in the proposal regarding what to > deprecated: The proposal of the KIP is to deprecate `retries` for producer, admin, and Streams. Maybe the c

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-18 Thread Guozhang Wang
Hi Matthias, Sorry for flooding the thread, but with this KIP I feel the design scope of https://issues.apache.org/jira/browse/KAFKA-6520 can be simplified a lot and may it the design can be just piggy-backed as part of this KIP, wdyt? Guozhang On Mon, May 18, 2020 at 9:47 AM Guozhang Wang wro

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-18 Thread Guozhang Wang
Hi Matthias, Just to add one more meta comment: for consumer, if it gets a TimeoutException polling records, would start timing all tasks since that single consumer would affect all tasks? For other blocking calls like `endOffsets()` etc, they are usually also issued on behalf of a batch of tasks,

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-18 Thread Bruno Cadonna
Hi Matthias, I am +1 (non-binding) on the KIP. Just one final remark: Wouldn't it be better to specify task.timeout.ms to -1 if no retry should be done? IMO it would make the config more intuitive because 0 would not have two possible meanings (i.e. try once and never try) anymore. Best, Bruno

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-16 Thread Guozhang Wang
Hello Matthias, Thanks for the updated KIP, overall I'm +1 on this proposal. Some minor comments (I know gmail mixed that again for me so I'm leaving it as a combo for both DISCUSS and VOTE :) 1) There are some inconsistent statements in the proposal regarding what to deprecated: at the beginning

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-15 Thread Boyang Chen
Good, good. Read through the discussions, the KIP looks good to me, +1 (non-binding) On Fri, May 15, 2020 at 11:51 AM Sophie Blee-Goldman wrote: > Called out! > > Seems like gmail struggles with [...] prefixed subjects. You'd think they > would adapt > all their practices to conform to the Apac

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-15 Thread Sophie Blee-Goldman
Called out! Seems like gmail struggles with [...] prefixed subjects. You'd think they would adapt all their practices to conform to the Apache Kafka mailing list standards, but no! +1 (non-binding) by the way On Fri, May 15, 2020 at 11:46 AM John Roesler wrote: > Hi Boyang, > > It is a separat

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-15 Thread John Roesler
Hi Boyang, It is a separate thread, and you have just revealed yourself as a gmail user ;) (Gmail sometimes conflates vote and discuss threads for no apparent reason ) -John On Fri, May 15, 2020, at 13:39, Boyang Chen wrote: > Hey Matthias, should this be on a separate VOTE thread? > > On Fri,

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-15 Thread Boyang Chen
Hey Matthias, should this be on a separate VOTE thread? On Fri, May 15, 2020 at 11:38 AM John Roesler wrote: > Thanks, Matthias! > > I’m +1 (binding) > > -John > > On Fri, May 15, 2020, at 11:55, Matthias J. Sax wrote: > > Hi, > > > > I would like to start the vote on KIP-572: > > > > > https://

Re: [VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-15 Thread John Roesler
Thanks, Matthias! I’m +1 (binding) -John On Fri, May 15, 2020, at 11:55, Matthias J. Sax wrote: > Hi, > > I would like to start the vote on KIP-572: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-572%3A+Improve+timeouts+and+retries+in+Kafka+Streams > > > -Matthias > > > Attachme

[VOTE] KIP-572: Improve timeouts and retires in Kafka Streams

2020-05-15 Thread Matthias J. Sax
Hi, I would like to start the vote on KIP-572: https://cwiki.apache.org/confluence/display/KAFKA/KIP-572%3A+Improve+timeouts+and+retries+in+Kafka+Streams -Matthias signature.asc Description: OpenPGP digital signature