Consumption on Partitions – 99,0 and 98 are getting stucked. Thanks Tushar From: "Mhaskar, Tushar" <tmhas...@paypal.com<mailto:tmhas...@paypal.com>> Date: Monday, May 2, 2016 at 9:52 PM To: "users@kafka.apache.org<mailto:users@kafka.apache.org>" <users@kafka.apache.org<mailto:users@kafka.apache.org>> Subject: MirrorMaker consumers getting stuck on certain partitions
Hi, I am running Mirror Maker (version 0.9 , new consumer). Sometimes I find that consumer gets stuck on certain partitions and offset doesn’t move in that case. I have 10 MM process running, each having 10 streams. The topic has 100 partitions. Below is the sample output of the consumer (I have cut short the output. Remaining partitions except the highlighted ones have similar lag in thousands). I can see the LOG size and Lag increasing but not the current offset and the MM is also running. GROUP, TOPIC, PARTITION, CURRENT OFFSET, LOG END OFFSET, LAG, OWNER slca.lvs.fpti.mm, fpti.platform.enrch, 65, 118463859, 118471286, 7427, slca.lvs.fpti.mm-6_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 68, 118490771, 118492211, 1440, slca.lvs.fpti.mm-6_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 26, 118484045, 118493447, 9402, slca.lvs.fpti.mm-2_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 14, 118492467, 118502925, 10458, slca.lvs.fpti.mm-1_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 99, 110230634, 118488448, 8257814, slca.lvs.fpti.mm-9_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 19, 118488594, 118497984, 9390, slca.lvs.fpti.mm-1_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 35, 118477757, 118478130, 373, slca.lvs.fpti.mm-3_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 80, 118472483, 118481154, 8671, slca.lvs.fpti.mm-8_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 21, 118478106, 118479540, 1434, slca.lvs.fpti.mm-2_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 59, 118487660, 118494558, 6898, slca.lvs.fpti.mm-5_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 48, 118483938, 118490119, 6181, slca.lvs.fpti.mm-4_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 95, 118490885, 118495214, 4329, slca.lvs.fpti.mm-9_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 58, 118499230, 118499608, 378, slca.lvs.fpti.mm-5_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 44, 118484130, 118491922, 7792, slca.lvs.fpti.mm-4_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 56, 118481454, 118485764, 4310, slca.lvs.fpti.mm-5_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 64, 118463461, 118471399, 7938, slca.lvs.fpti.mm-6_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 81, 118481575, 118482081, 506, slca.lvs.fpti.mm-8_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 0, 116010184, 118476683, 2466499, slca.lvs.fpti.mm-0_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 40, 118478651, 118486854, 8203, slca.lvs.fpti.mm-4_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 12, 118473021, 118481007, 7986, slca.lvs.fpti.mm-1_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 30, 118462885, 118470919, 8034, slca.lvs.fpti.mm-3_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 98, 117227057, 118484055, 1256998, slca.lvs.fpti.mm-9_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 36, 118491829, 118498669, 6840, slca.lvs.fpti.mm-3_/10.196.246.38 GROUP, TOPIC, PARTITION, CURRENT OFFSET, LOG END OFFSET, LAG, OWNER slca.lvs.fpti.mm, fpti.platform.enrch, 65, 118483038, 118489002, 5964, slca.lvs.fpti.mm-6_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 68, 118500463, 118510063, 9600, slca.lvs.fpti.mm-6_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 26, 118503344, 118511243, 7899, slca.lvs.fpti.mm-2_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 14, 118514764, 118520550, 5786, slca.lvs.fpti.mm-1_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 99, 110230634, 118506337, 8275703, slca.lvs.fpti.mm-9_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 19, 118507871, 118515735, 7864, slca.lvs.fpti.mm-1_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 35, 118487244, 118495868, 8624, slca.lvs.fpti.mm-3_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 80, 118491768, 118499046, 7278, slca.lvs.fpti.mm-8_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 9, 118498940, 118499194, 254, slca.lvs.fpti.mm-0_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 6, 118516902, 118517425, 523, slca.lvs.fpti.mm-0_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 61, 118514062, 118521457, 7395, slca.lvs.fpti.mm-6_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 0, 116100206, 118494575, 2394369, slca.lvs.fpti.mm-0_/10.196.246.37 slca.lvs.fpti.mm, fpti.platform.enrch, 40, 118497705, 118504514, 6809, slca.lvs.fpti.mm-4_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 12, 118492007, 118498617, 6610, slca.lvs.fpti.mm-1_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 30, 118481808, 118488785, 6977, slca.lvs.fpti.mm-3_/10.196.246.38 slca.lvs.fpti.mm, fpti.platform.enrch, 98, 117318343, 118501939, 1183596, slca.lvs.fpti.mm-9_/10.196.246.37 Anyone else facing the same problem? Thanks Tushar
