We already did some improvements for 0.10.2 that was release last week. We plan more improvements for 0.10.3/0.11.0 (whatever the next release number will be).
Feedback like this is very valuable to us! We are aware of some issues, but rely on people to report problems and ideas on how to improve the behavior. Really appreciate your input! -Matthias On 2/28/17 9:34 AM, Nicolas Fouché wrote: > Thanks Mathias. I read a few weeks ago that there was some enhancements > planned on the reblancing feature for the next release. It would be a great > addition ! > > -Nicolas > > > 2017-02-28 18:15 GMT+01:00 Matthias J. Sax <matth...@confluent.io>: > >> Hi Nicolas, >> >> an optimization like this would make a lot of sense. We did have some >> discussions around this already. However, it's more tricky to do than it >> seems at a first glace. We hope to introduce something like this for the >> next release. >> >> >> -Matthias >> >> >> >> On 2/28/17 9:10 AM, Nicolas Fouché wrote: >>> Hi, >>> >>> I have 10 Kafka Streams processes which consume a topic with 10 >> partitions, >>> with a few changelog topics. >>> >>> Let's say these processes are all stopped, and I start them nearly at the >>> same time (in a matter of seconds). >>> >>> The first process seems to start initializing all state stores, which >> takes >>> 15 to 30 minutes in my case. In the meantime all other processes wait >> with >>> "(Re-)joining group xyz". >>> >>> When the first process finishes its initialization, everyone joins, and >>> everyone (including the first process) initializes state stores again. >>> >>> I wondered: what if the first process would wait a few minutes, to see if >>> others processes arrive in the group ? So no re-distribution would be >>> needed. >>> >>> - Nicolas >>> >> >> >
signature.asc
Description: OpenPGP digital signature
