[DISCUSS] KIP-287: Support partition and processor expansion for stateful processing jobs

Hi all, I have created KIP-287 to provide high level design to support partition and processor expansion for stateful processing. See https://cwiki.apache.org/confluence/display/KAFKA/KIP-287%3A+Support+partition+and+processor+expansion+for+stateful+processing+jobs . Instead of planning to finish

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

Thanks for the notes by Jun and Ray. I have read through the notes. It seems that there are a few questions for the alternative solution by Jan and maybe Jan will answer these questions later? I have summarized the solution, which I previously provided in this thread, in KIP-287, to hopefully show

Re: [DISCUSS] KIP-276: Add StreamsConfig prefix for different consumers

Thank you Guozhang for the explanation. Ewen, I think your concern has been addressed, do you want share any further concern on this KIP? Thank you! From: Guozhang Wang Sent: Friday, April 13, 2018 12:34 AM To: dev@kafka.apache.org Subject: Re: [DISCUSS] KIP-27

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

Hi, I had a question from Monday's meeting. The current mechanism, as Ray's notes points out, is to create a copy consumer in which a switch over happens when it catches up. Meanwhile, it looks like a producer would still be writing using the old partitioning scheme. Wouldn't there be a case where

[VOTE] KIP-279: Fix log divergence between leader and follower after fast leader fail over

Hi All, I would like to start the vote on KIP-279: Fix log divergence between leader and follower after fast leader fail over. For reference, here's the KIP wiki: https://cwiki.apache.org/confluence/display/KAFKA/KIP-279%3A+Fix+log+divergence+between+leader+and+follower+after+fast+leader+fail+o

Contributor List Request

Hi, May I please be added to the contributor list, according to: https://kafka.apache.org/contributing.html Please let me know once I'm added, and I'll start picking up a starter bug on the JIRA board. Thanks! Abhishek

Re: [VOTE] KIP-279: Fix log divergence between leader and follower after fast leader fail over

+1 On Sat, Apr 14, 2018 at 3:54 PM, Anna Povzner wrote: > Hi All, > > > I would like to start the vote on KIP-279: Fix log divergence between > leader and follower after fast leader fail over. > > > For reference, here's the KIP wiki: > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 27

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

Hey Jeff, Thanks for the review. The scheme for expanding processors of the stateful processing job is described in "Support processor expansion" section in KIP-287 (link

Re: [DISCUSS] KIP-269: Substitution Within Configuration Values

I am unsure if substitution should be supported for just JAAS configs or if we should allow it for cluster/broker/consumer configs. What I think would be helpful would be to boil down this proposal to its most essential requirement, and I think the discussion has helped us arrive at what that look

[jira] [Created] (KAFKA-6790) Missing documentation for processor nodes on Kafka Streams

robin m created KAFKA-6790: -- Summary: Missing documentation for processor nodes on Kafka Streams Key: KAFKA-6790 URL: https://issues.apache.org/jira/browse/KAFKA-6790 Project: Kafka Issue Type: Bug