Idriss Neumann created KAFKA-6191:
-------------------------------------
Summary: Consumers are staying blocked on rebalancing
Key: KAFKA-6191
URL: https://issues.apache.org/jira/browse/KAFKA-6191
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 0.10.2.1
Reporter: Idriss Neumann
My logstash's consumers are staying blocked since couple days with a warn
"group 'logstash' is rebalancing". Here is the output of the "describe" command
:
{code}[root@host1 ~]# /opt/kafka/bin/kafka-consumer-groups.sh
--bootstrap-server host1:9092,host2:9092,host3:9092 --group logstash --describe
Note: This will only show information about consumers that use the Java
consumer API (non-ZooKeeper-based consumers).
Warning: Consumer group 'logstash' is rebalancing.
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
CONSUMER-ID HOST
CLIENT-ID
- - - - -
logstash-4-faf1109b-8bec-41f7-b883-e422ebfa33a3 /x.x.x.233
logstash-4
- - - - -
logstash-3-25b0380b-410e-42b6-8911-aab1a5435d48 /x.x.x.233
logstash-3
- - - - -
logstash-2-9667ad4a-59aa-47e3-8633-51f3a9f13bb8 /x.x.x.233
logstash-2
- - - - -
logstash-0-b205b42e-06ec-4d48-8edc-17c7d750fd66 /x.x.x.233
logstash-0
- - - - -
logstash-1-990c6639-f8af-4476-a8cd-9a38c7ebd355 /x.x.x.233
logstash-1
rsyslog 5 0 120815
120815 - -
-
collectd 8 29828791 42691214
12862423 - -
-
collectd 4 29796008 42898658
13102650 - -
-
rsyslog 9 0 121227
121227 - -
-
collectd 7 29752981 42739230
12986249 - -
-
rsyslog 1 0 120739
120739 - -
-
collectd 1 29779498 42783100
13003602 - -
-
rsyslog 4 0 121258
121258 - -
-
collectd 3 29839893 42688409
12848516 - -
-
rsyslog 2 0 120306
120306 - -
-
collectd 6 29795665 42742170
12946505 - -
-
rsyslog 0 0 120045
120045 - -
-
collectd 0 29805810 42883069
13077259 - -
-
rsyslog 8 0 121075
121075 - -
-
rsyslog 7 0 120463
120463 - -
-
rsyslog 3 0 120773
120773 - -
-
collectd 9 29803212 42631849
12828637 - -
-
collectd 2 29810407 42654801
12844394 - -
-
rsyslog 6 0 120381
120381 - -
-
collectd 5 29803946 42793991
12990045 - -
-{code}
If I change the group id in the logstash conf, the consumers are working fine
and consume messages of topics from the beginning:
{code}[root@host1 ~]# /opt/kafka/bin/kafka-consumer-groups.sh
--bootstrap-server host1:9092,host2:9092,host3:9092 --group logstash_new
--describe
Note: This will only show information about consumers that use the Java
consumer API (non-ZooKeeper-based consumers).
TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG
CONSUMER-ID HOST
CLIENT-ID
rsyslog 4 121276 121281 5
logstash-2-234ee9b1-7f98-4e04-b346-823b11cd9327 /x.x.x.228
logstash-2
collectd 4 42899540 42899990 450
logstash-2-234ee9b1-7f98-4e04-b346-823b11cd9327 /x.x.x.228
logstash-2
rsyslog 5 120831 120836 5
logstash-2-234ee9b1-7f98-4e04-b346-823b11cd9327 /x.x.x.228
logstash-2
collectd 5 42794913 42795251 338
logstash-2-234ee9b1-7f98-4e04-b346-823b11cd9327 /x.x.x.228
logstash-2
rsyslog 6 120397 120402 5
logstash-3-ca37feda-88d6-4d23-b193-4570fd02d991 /x.x.x.228
logstash-3
collectd 6 42743287 42743428 141
logstash-3-ca37feda-88d6-4d23-b193-4570fd02d991 /x.x.x.228
logstash-3
rsyslog 7 120484 120491 7
logstash-3-ca37feda-88d6-4d23-b193-4570fd02d991 /x.x.x.228
logstash-3
collectd 7 42740445 42740611 166
logstash-3-ca37feda-88d6-4d23-b193-4570fd02d991 /x.x.x.228
logstash-3
rsyslog 8 121089 121096 7
logstash-4-54c81fe7-46d3-4541-b358-edc2d22ee877 /x.x.x.228
logstash-4
collectd 8 42692173 42692546 373
logstash-4-54c81fe7-46d3-4541-b358-edc2d22ee877 /x.x.x.228
logstash-4
rsyslog 9 121245 121249 4
logstash-4-54c81fe7-46d3-4541-b358-edc2d22ee877 /x.x.x.228
logstash-4
collectd 9 42632827 42633201 374
logstash-4-54c81fe7-46d3-4541-b358-edc2d22ee877 /x.x.x.228
logstash-4
rsyslog 0 120066 120070 4
logstash-0-d9675b44-b1dc-4a66-8513-5af2cd0f68fd /x.x.x.228
logstash-0
collectd 0 42884280 42884455 175
logstash-0-d9675b44-b1dc-4a66-8513-5af2cd0f68fd /x.x.x.228
logstash-0
rsyslog 1 120763 120766 3
logstash-0-d9675b44-b1dc-4a66-8513-5af2cd0f68fd /x.x.x.228
logstash-0
collectd 1 42784306 42784468 162
logstash-0-d9675b44-b1dc-4a66-8513-5af2cd0f68fd /x.x.x.228
logstash-0
rsyslog 2 120325 120333 8
logstash-1-81b917fd-9732-4030-89b5-5d20681fc7a0 /x.x.x.228
logstash-1
collectd 2 42655705 42656061 356
logstash-1-81b917fd-9732-4030-89b5-5d20681fc7a0 /x.x.x.228
logstash-1
rsyslog 3 120789 120803 14
logstash-1-81b917fd-9732-4030-89b5-5d20681fc7a0 /x.x.x.228
logstash-1
collectd 3 42689329 42689665 336
logstash-1-81b917fd-9732-4030-89b5-5d20681fc7a0 /x.x.x.228
logstash-1{code}
The logstash configuration :
{code}input {
kafka {
topics_pattern => 'collectd|rsyslog'
group_id => 'logstash'
bootstrap_servers => "host1:9092,host2:9092,host3:9092"
consumer_threads => 5
add_field => {
from => "lst1"
}
codec => json
}
}{code}
Thanks in advance for any help.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
