Hi Sameer, Could you elaborate on your question? Are you concerned that machine2 does not have any tasks in the beginning?
Could you share your streams configuration? In particular how many threads does each stream instance have? Also how many topics and partitions do you have? Thanks, Eno > On 23 Jun 2017, at 17:31, Sameer Kumar <sam.kum.w...@gmail.com> wrote: > > Hi, > > Came across a rebalancing issue in using KafkaStreams. I have two machines, > Machine1 and Machine2, machine1 is consuming all partitions and machine2 is > completely free and not processing any partitions. If I shutdown machine1, > then machine2 will take over and would start consuming all partitions. > > But, in this scenario, its not using the complete cluster. > > *Machine 1* > 2017-06-23 21:54:07 INFO KafkaStreams:224 - stream-client > [LIC3-43-2b05186c-cbf2-4f9c-acc4-0dd929c7d647] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:07 INFO StreamThread:858 - stream-thread > [StreamThread-27] Creating active task 1_2 with assigned partitions > [LIC3-43-lic3-deb-ci2-43-repartition-2, > LIC3-43-lic3-cnt-ci-43-repartition-2] > 2017-06-23 21:54:07 INFO StreamThread:163 - stream-thread > [StreamThread-20] State transition from PARTITIONS_REVOKED to > ASSIGNING_PARTITIONS. > 2017-06-23 21:54:07 INFO KafkaStreams:224 - stream-client > [LIC3-43-2b05186c-cbf2-4f9c-acc4-0dd929c7d647] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:07 INFO StreamThread:163 - stream-thread > [StreamThread-31] State transition from PARTITIONS_REVOKED to > ASSIGNING_PARTITIONS. > 2017-06-23 21:54:07 INFO StreamThread:163 - stream-thread > [StreamThread-13] State transition from PARTITIONS_REVOKED to > ASSIGNING_PARTITIONS. > 2017-06-23 21:54:07 INFO KafkaStreams:224 - stream-client > [LIC3-43-2b05186c-cbf2-4f9c-acc4-0dd929c7d647] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:07 INFO KafkaStreams:224 - stream-client > [LIC3-43-2b05186c-cbf2-4f9c-acc4-0dd929c7d647] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:07 INFO StreamThread:163 - stream-thread > [StreamThread-16] State transition from ASSIGNING_PARTITIONS to RUNNING. > 2017-06-23 21:54:07 INFO StreamThread:858 - stream-thread > [StreamThread-18] Creating active task 0_4 with assigned partitions > [testS5-4] > 2017-06-23 21:54:07 INFO StreamThread:858 - stream-thread > [StreamThread-13] Creating active task 0_2 with assigned partitions > [testS5-2] > 2017-06-23 21:54:07 INFO KafkaStreams:224 - stream-client > [LIC3-43-2b05186c-cbf2-4f9c-acc4-0dd929c7d647] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:07 INFO KafkaStreams:224 - stream-client > [LIC3-43-2b05186c-cbf2-4f9c-acc4-0dd929c7d647] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:07 INFO StreamThread:858 - stream-thread > [StreamThread-31] Creating active task 2_0 with assigned partitions > [LIC3-43-lic3-cnt-li-43-repartition-0, > LIC3-43-lic3-cnt-li-43_1-repartition-0] > 2017-06-23 21:54:07 INFO StreamThread:858 - stream-thread > [StreamThread-20] Creating active task 0_9 with assigned partitions > [testS5-9] > 2017-06-23 21:54:07 INFO KafkaStreams:224 - stream-client > [LIC3-43-2b05186c-cbf2-4f9c-acc4-0dd929c7d647] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:07 INFO KafkaStreams:224 - stream-client > [LIC3-43-2b05186c-cbf2-4f9c-acc4-0dd929c7d647] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:07 INFO StreamThread:858 - stream-thread > [StreamThread-29] Creating active task 2_5 with assigned partitions > [LIC3-43-lic3-cnt-li-43-repartition-5, > LIC3-43-lic3-cnt-li-43_1-repartition-5] > > > *Machine2* > > 21:54:08 INFO StreamThread:248 - stream-thread [StreamThread-8] at state > RUNNING: partitions [] revoked at the beginning of consumer rebalance. > 2017-06-23 21:54:08 INFO AbstractCoordinator:420 - (Re-)joining group > LIC3-43 > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-5] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1049 - stream-thread > [StreamThread-5] Removing all active tasks [] > 2017-06-23 21:54:08 INFO KafkaStreams:224 - stream-client > [LIC3-43-47a995d4-2f53-4837-8b8a-c5550bc7b3eb] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:08 INFO StreamThread:1064 - stream-thread > [StreamThread-5] Removing all standby tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-18] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:163 - stream-thread [StreamThread-8] > State transition from RUNNING to PARTITIONS_REVOKED. > 2017-06-23 21:54:08 INFO StreamThread:1049 - stream-thread > [StreamThread-18] Removing all active tasks [] > 2017-06-23 21:54:08 INFO AbstractCoordinator:420 - (Re-)joining group > LIC3-43 > 2017-06-23 21:54:08 INFO KafkaStreams:224 - stream-client > [LIC3-43-47a995d4-2f53-4837-8b8a-c5550bc7b3eb] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:08 INFO StreamThread:1064 - stream-thread > [StreamThread-18] Removing all standby tasks [] > 2017-06-23 21:54:08 INFO KafkaStreams:224 - stream-client > [LIC3-43-47a995d4-2f53-4837-8b8a-c5550bc7b3eb] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-1] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-15] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1049 - stream-thread > [StreamThread-15] Removing all active tasks [] > 2017-06-23 21:54:08 INFO KafkaStreams:224 - stream-client > [LIC3-43-47a995d4-2f53-4837-8b8a-c5550bc7b3eb] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:08 INFO KafkaStreams:224 - stream-client > [LIC3-43-47a995d4-2f53-4837-8b8a-c5550bc7b3eb] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:08 INFO AbstractCoordinator:420 - (Re-)joining group > LIC3-43 > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-30] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-14] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1064 - stream-thread > [StreamThread-15] Removing all standby tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1049 - stream-thread > [StreamThread-1] Removing all active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1064 - stream-thread > [StreamThread-1] Removing all standby tasks [] > 2017-06-23 21:54:08 INFO AbstractCoordinator:420 - (Re-)joining group > LIC3-43 > 2017-06-23 21:54:08 INFO AbstractCoordinator:420 - (Re-)joining group > LIC3-43 > 2017-06-23 21:54:08 INFO StreamThread:1049 - stream-thread > [StreamThread-14] Removing all active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1049 - stream-thread > [StreamThread-30] Removing all active tasks [] > 2017-06-23 21:54:08 INFO KafkaStreams:224 - stream-client > [LIC3-43-47a995d4-2f53-4837-8b8a-c5550bc7b3eb] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:08 INFO StreamThread:1064 - stream-thread > [StreamThread-30] Removing all standby tasks [] > 2017-06-23 21:54:08 INFO AbstractCoordinator:420 - (Re-)joining group > LIC3-43 > 2017-06-23 21:54:08 INFO StreamThread:1064 - stream-thread > [StreamThread-14] Removing all standby tasks [] > 2017-06-23 21:54:08 INFO KafkaStreams:224 - stream-client > [LIC3-43-47a995d4-2f53-4837-8b8a-c5550bc7b3eb] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:08 INFO KafkaStreams:224 - stream-client > [LIC3-43-47a995d4-2f53-4837-8b8a-c5550bc7b3eb] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-20] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1049 - stream-thread > [StreamThread-20] Removing all active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-6] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO KafkaStreams:224 - stream-client > [LIC3-43-47a995d4-2f53-4837-8b8a-c5550bc7b3eb] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-9] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO AbstractCoordinator:420 - (Re-)joining group > LIC3-43 > 2017-06-23 21:54:08 INFO StreamThread:1049 - stream-thread > [StreamThread-9] Removing all active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1064 - stream-thread > [StreamThread-9] Removing all standby tasks [] > 2017-06-23 21:54:08 INFO AbstractCoordinator:420 - (Re-)joining group > LIC3-43 > 2017-06-23 21:54:08 INFO KafkaStreams:224 - stream-client > [LIC3-43-47a995d4-2f53-4837-8b8a-c5550bc7b3eb] State transition from > REBALANCING to REBALANCING. > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-4] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1042 - stream-thread > [StreamThread-31] Updating suspended tasks to contain active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1049 - stream-thread > [StreamThread-31] Removing all active tasks [] > 2017-06-23 21:54:08 INFO StreamThread:1064 - stream-thread > [StreamThread-31] Removing all standby tasks [] > > > Regards, > -Sameer.
