Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2020-05-08 Thread Matthias J. Sax
Thanks John! Makes sense. On 5/4/20 10:00 AM, Guozhang Wang wrote: > Thanks for the explanation John. > > > Guozhang > > On Sun, May 3, 2020 at 7:10 PM John Roesler wrote: > >> Hi Guozhang, >> >> Ah, good question. Yes, the assignor will always now try to achieve a >> perfect balance. This wa

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2020-05-04 Thread Guozhang Wang
Thanks for the explanation John. Guozhang On Sun, May 3, 2020 at 7:10 PM John Roesler wrote: > Hi Guozhang, > > Ah, good question. Yes, the assignor will always now try to achieve a > perfect balance. This was also the proposed default in the KIP before. The > config would have allowed users t

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2020-05-03 Thread John Roesler
Hi Guozhang, Ah, good question. Yes, the assignor will always now try to achieve a perfect balance. This was also the proposed default in the KIP before. The config would have allowed users to relax the search for perfection. This is actually one of our motivations now to remove it. We feel it’

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2020-05-03 Thread Guozhang Wang
Hello John / Sophie: With this config removed, would the assignor always tries to to achieve the "perfect balance" (of course, it may be a sub-optimal local plateau) or there's an internal hard-coded factor to still retain some satisfying threshold? Guozhang On Sun, May 3, 2020 at 9:23 AM John R

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2020-05-03 Thread John Roesler
Hi Matthias, We originally proposed that config to allow us to skip migrating tasks if the current balance is “good enough”. But during implementation, we became concerned that supporting this option increased code complexity, and it’s also an extra concept for users to have to learn. To keep

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2020-05-01 Thread Matthias J. Sax
Can you elaborate why to remove it? On 5/1/20 11:29 AM, Sophie Blee-Goldman wrote: > Hey all, > > We'd like to make a slight modification to the proposal in this KIP and > remove > the *balance.factor* config. We will update the KIP accordingly. Please let > us know > if you have any concerns. >

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2020-05-01 Thread Sophie Blee-Goldman
Hey all, We'd like to make a slight modification to the proposal in this KIP and remove the *balance.factor* config. We will update the KIP accordingly. Please let us know if you have any concerns. Cheers, Sophie On Wed, Jan 15, 2020 at 12:48 PM John Roesler wrote: > Hello all, > > After a lon

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2020-01-15 Thread John Roesler
Hello all, After a long hiatus, I've just realized that I'm now able to upgrade my non-binding support to a binding +1 for KIP-441. This brings the vote tally to: 3 binding +1s: Guozhang, Bill, and myself 3 non-binding +1s: Bruno, Vinoth, and Sophie Since the vote has been open for at least 72

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2019-10-28 Thread John Roesler
Hey all, Now that the 2.4 release storm is over, I'd like to bump this vote thread. Currently, we have two binding +1s (Guozhang and Bill), and four non-binding ones (Bruno, Vinoth, Sophie, and myself), and no vetoes. Thanks, -John On Thu, Sep 12, 2019 at 12:54 PM Bill Bejeck wrote: > > +1 (bi

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2019-09-12 Thread Bill Bejeck
+1 (binding) On Thu, Sep 12, 2019 at 1:53 PM Sophie Blee-Goldman wrote: > +1 (non-binding) > > On Wed, Sep 11, 2019 at 11:38 AM Vinoth Chandar > wrote: > > > +1 (non-binding). > > > > On Fri, Sep 6, 2019 at 12:46 AM Bruno Cadonna > wrote: > > > > > +1 (non-binding) > > > > > > On Fri, Sep 6, 2

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2019-09-12 Thread Sophie Blee-Goldman
+1 (non-binding) On Wed, Sep 11, 2019 at 11:38 AM Vinoth Chandar wrote: > +1 (non-binding). > > On Fri, Sep 6, 2019 at 12:46 AM Bruno Cadonna wrote: > > > +1 (non-binding) > > > > On Fri, Sep 6, 2019 at 12:32 AM Guozhang Wang > wrote: > > > > > > +1 (binding). > > > > > > On Thu, Sep 5, 2019 a

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2019-09-11 Thread Vinoth Chandar
+1 (non-binding). On Fri, Sep 6, 2019 at 12:46 AM Bruno Cadonna wrote: > +1 (non-binding) > > On Fri, Sep 6, 2019 at 12:32 AM Guozhang Wang wrote: > > > > +1 (binding). > > > > On Thu, Sep 5, 2019 at 2:47 PM John Roesler wrote: > > > > > Hello, all, > > > > > > After a great discussion, I'd li

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2019-09-06 Thread Bruno Cadonna
+1 (non-binding) On Fri, Sep 6, 2019 at 12:32 AM Guozhang Wang wrote: > > +1 (binding). > > On Thu, Sep 5, 2019 at 2:47 PM John Roesler wrote: > > > Hello, all, > > > > After a great discussion, I'd like to open voting on KIP-441, > > to avoid long restore times in Streams after rebalancing. > >

Re: [VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2019-09-05 Thread Guozhang Wang
+1 (binding). On Thu, Sep 5, 2019 at 2:47 PM John Roesler wrote: > Hello, all, > > After a great discussion, I'd like to open voting on KIP-441, > to avoid long restore times in Streams after rebalancing. > Please cast your votes! > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-441:+

[VOTE] KIP-441: Smooth Scaling Out for Kafka Streams

2019-09-05 Thread John Roesler
Hello, all, After a great discussion, I'd like to open voting on KIP-441, to avoid long restore times in Streams after rebalancing. Please cast your votes! https://cwiki.apache.org/confluence/display/KAFKA/KIP-441:+Smooth+Scaling+Out+for+Kafka+Streams Thanks, -John