[
https://issues.apache.org/jira/browse/KAFKA-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14322765#comment-14322765
]
Beletsky Andrey commented on KAFKA-1958:
----------------------------------------
+Reason of it:+
{code}
ZookeeperConsumerConnector.shutdown() ->
ConsumerFetcherManager.stopConnections() ->
leaderFinderThread.shutdown() - fails with
InterruptedException
{code}
Stack trace:
{noformat}
java.lang.InterruptedException: null
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1301)
~[na:1.7.0_65]
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:236)
~[na:1.7.0_65]
at kafka.utils.ShutdownableThread.shutdown(ShutdownableThread.scala:36)
~[kafka_2.10-0.8.1.1.jar:na]
at
kafka.consumer.ConsumerFetcherManager.stopConnections(ConsumerFetcherManager.scala:143)
~[kafka_2.10-0.8.1.1.jar:na]
at
kafka.consumer.ZookeeperConsumerConnector.liftedTree1$1(ZookeeperConsumerConnector.scala:171)
[kafka_2.10-0.8.1.1.jar:na]
at
kafka.consumer.ZookeeperConsumerConnector.shutdown(ZookeeperConsumerConnector.scala:167)
[kafka_2.10-0.8.1.1.jar:na]
at
kafka.javaapi.consumer.ZookeeperConsumerConnector.shutdown(ZookeeperConsumerConnector.scala:110)
[kafka_2.10-0.8.1.1.jar:na]
{noformat}
I think ZookeeperConsumerConnector should be smart and correctly close own
resources (threads), release partition ownership, remove consumer Id from
/consumer/myGroup/ids/<consumer_id>.
> ZookeeperConsumerConnector doesn't remove consumer node on shutdown.
> --------------------------------------------------------------------
>
> Key: KAFKA-1958
> URL: https://issues.apache.org/jira/browse/KAFKA-1958
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.8.1.1
> Reporter: Beletsky Andrey
> Assignee: Neha Narkhede
> Labels: consumer, shutdown, zookeeper
>
> We use kafka with ZooKeeper via high level consumer.
> There is a scheduled job that creates a consumer with specific group, does
> necessary logic and shuts down this consumer.
> +An issue:+
> Nobody deletes */consumers/myGroup/ids/myGroup_<ip>_<postfix>*. And after
> several job runs there are a lot of dead consumer IDs under myGroup. I've got
> [an
> issue|https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whysomeoftheconsumersinaconsumergroupneverreceiveanymessage?]
> that new consumer doesn't see a partition.
> We start to implement an approach to remove a consumer nodes from Zookeeper
> manually after consumer is shutted down.
> I think better way to remove this node during
> *ZookeeperConsumerConnector.shutdown()*.
> *P.S.:*
> If I missed something in your sources please let me know.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
