Are you building with any released versions of Kafka or with a build from Kafka trunk? There are a few fixes we have made post 0.10.2 in trunk that has largely reduced the rebalance latency so I'd recommend try using a build from Kafka trunk for testing if possible.
Guozhang On Wed, Mar 15, 2017 at 10:46 AM, Tianji Li <skyah...@gmail.com> wrote: > It seems independent to the rocksdb sizes. It also took 5 minutes when > there are 375Mbytes this morning... > > On Wed, Mar 15, 2017 at 9:13 AM, Sachin Mittal <sjmit...@gmail.com> wrote: > > > rocksdb state store initialization may be taking up that time. > > Whats the size of your rockksdb state directory. May be partitioning the > > source topic, increasing the number of threads/instances processing the > > source and reducing the time window of aggregation can help in reducing > the > > startup time. > > > > > > > > On Wed, Mar 15, 2017 at 6:36 PM, Tianji Li <skyah...@gmail.com> wrote: > > > > > Hi there, > > > > > > In the experiments I am doing now, if I restart the streams > application, > > I > > > have to wait for around 5 minutes for some reason. > > > > > > I can see something in the Kafka logs: > > > > > > [2017-03-15 08:36:18,118] INFO [GroupCoordinator 0]: Preparing to > > > restabilize group xxx-test25 with old generation 2 > > > (kafka.coordinator.GroupCoordinator) > > > [2017-03-15 08:41:08,449] INFO [GroupCoordinator 0]: Stabilized group > > > xxx-test25 generation 3 (kafka.coordinator.GroupCoordinator) > > > [2017-03-15 08:41:08,473] INFO [GroupCoordinator 0]: Assignment > received > > > from leader for group xxx-test25 for generation 3 > > > (kafka.coordinator.GroupCoordinator) > > > > > > What is happening in these 5 minutes? How to reduce it? > > > > > > Thanks > > > Tianji > > > > > > -- -- Guozhang
