Thanks Mathias. I read a few weeks ago that there was some enhancements planned on the reblancing feature for the next release. It would be a great addition !
-Nicolas 2017-02-28 18:15 GMT+01:00 Matthias J. Sax <matth...@confluent.io>: > Hi Nicolas, > > an optimization like this would make a lot of sense. We did have some > discussions around this already. However, it's more tricky to do than it > seems at a first glace. We hope to introduce something like this for the > next release. > > > -Matthias > > > > On 2/28/17 9:10 AM, Nicolas Fouché wrote: > > Hi, > > > > I have 10 Kafka Streams processes which consume a topic with 10 > partitions, > > with a few changelog topics. > > > > Let's say these processes are all stopped, and I start them nearly at the > > same time (in a matter of seconds). > > > > The first process seems to start initializing all state stores, which > takes > > 15 to 30 minutes in my case. In the meantime all other processes wait > with > > "(Re-)joining group xyz". > > > > When the first process finishes its initialization, everyone joins, and > > everyone (including the first process) initializes state stores again. > > > > I wondered: what if the first process would wait a few minutes, to see if > > others processes arrive in the group ? So no re-distribution would be > > needed. > > > > - Nicolas > > > >
