[
https://issues.apache.org/jira/browse/KAFKA-1958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Beletsky Andrey updated KAFKA-1958:
-----------------------------------
Description:
We use kafka with ZooKeeper via high level consumer.
There is a scheduled job that creates a consumer with specific group, does
necessary logic and shuts down this consumer.
+An issue:+
Nobody deletes */consumers/myGroup/ids/myGroup_<ip>_<postfix>*. And after
several job runs there are a lot of dead consumer IDs under myGroup. I've got
[an
issue|https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whysomeoftheconsumersinaconsumergroupneverreceiveanymessage?]
that new consumer doesn't see a partition.
We start to implement an approach to remove a consumer nodes from Zookeeper
manually after consumer is shutted down.
I think better way to remove this node during
*ZookeeperConsumerConnector.shutdown()*.
*P.S.:*
If I missed something in your sources please let me know.
was:
We use kafka with ZooKeeper via high level consumer.
There is a scheduled job that creates a consumer with specific group, does
necessary logic and shuts down this consumer.
An issue:
Nobody deletes */consumers/myGroup/ids/myGroup_<ip>_<postfix>*. And after
several job runs there are a lot of dead consumer IDs under myGroup. I've got
[an
issue|https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whysomeoftheconsumersinaconsumergroupneverreceiveanymessage?]
that new consumer doesn't see a partition.
We start to implement an approach to remove a consumer nodes from Zookeeper
manually after consumer is shutted down.
I think better way to remove this node during
*ZookeeperConsumerConnector.shutdown()*.
*P.S.:*
If I missed something in your sources please let me know.
> ZookeeperConsumerConnector doesn't remove consumer node on shutdown.
> --------------------------------------------------------------------
>
> Key: KAFKA-1958
> URL: https://issues.apache.org/jira/browse/KAFKA-1958
> Project: Kafka
> Issue Type: Bug
> Components: consumer
> Affects Versions: 0.8.1.1
> Reporter: Beletsky Andrey
> Assignee: Neha Narkhede
> Labels: consumer, shutdown, zookeeper
>
> We use kafka with ZooKeeper via high level consumer.
> There is a scheduled job that creates a consumer with specific group, does
> necessary logic and shuts down this consumer.
> +An issue:+
> Nobody deletes */consumers/myGroup/ids/myGroup_<ip>_<postfix>*. And after
> several job runs there are a lot of dead consumer IDs under myGroup. I've got
> [an
> issue|https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whysomeoftheconsumersinaconsumergroupneverreceiveanymessage?]
> that new consumer doesn't see a partition.
> We start to implement an approach to remove a consumer nodes from Zookeeper
> manually after consumer is shutted down.
> I think better way to remove this node during
> *ZookeeperConsumerConnector.shutdown()*.
> *P.S.:*
> If I missed something in your sources please let me know.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
