Michael Golovanov created KAFKA-6537:
----------------------------------------
Summary: Duplicate consumers after consumer group rebalance
Key: KAFKA-6537
URL: https://issues.apache.org/jira/browse/KAFKA-6537
Project: Kafka
Issue Type: Bug
Components: consumer
Affects Versions: 0.10.1.1
Reporter: Michael Golovanov
*_Deployment description_*
Kafka brokers have been deployed on ten (10) nodes. Zookeeper cluster have
seven (7) nodes. Nodes of Kafka brokers shared with Zookeeper nodes on bare
metal hosts.
Broker/Zookeeper hosts OS is Redhat 7 and JVM version is Java 8. Host names are
grid48, grid237 и grid251.
We have one topic with six (6) patitions. Kafka consumers deployed on three (3)
hosts. Each host have two (2) consumers. All consumers belong to single group.
*_Error description_*
After start all consumers Apache Kafka partitions of topic was balanced evenly.
grid237 owns partitions 0,1 (0 - consumer thread-0, 1, consumer thread-1)
grid251 owns partitions 2,3 (2 - consumer thread-0, consumer thread-1)
grid48 owns partitions 4,5 (4- consumer thread-0, 5, consumer thread-1)
After some period of time we see haotic revokes and assigns partitions between
brokers and then all partitions in log assigned to one consumer on one node
grid48
But really all partitions read not only by thread-1 consumer, but thread-0 on
grid48. And all messages from topic partitions was duplicate. Consumer thread-0
try to commit message offset and get commit error, thread-1 consumer
successfully commit offset.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
