[
https://issues.apache.org/jira/browse/KAFKA-2948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15047782#comment-15047782
]
ASF GitHub Bot commented on KAFKA-2948:
---------------------------------------
GitHub user rajinisivaram opened a pull request:
https://github.com/apache/kafka/pull/645
KAFKA-2948: Remove unused topics from producer metadata set
If no messages are sent to a topic during the last refresh interval or if
UNKNOWN_TOPIC_OR_PARTITION error is received, remove the topic from the
metadata list. Topics are added to the list on the next attempt to send a
message to the topic.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/rajinisivaram/kafka KAFKA-2948
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/645.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #645
----
commit f7e40e5ce515d700e8cc7ab02a0f16141fa14f67
Author: rsivaram <rsiva...@uk.ibm.com>
Date: 2015-12-09T00:16:18Z
KAFKA-2948: Remove unused topics from producer metadata set
----
> Kafka producer does not cope well with topic deletions
> ------------------------------------------------------
>
> Key: KAFKA-2948
> URL: https://issues.apache.org/jira/browse/KAFKA-2948
> Project: Kafka
> Issue Type: Bug
> Components: producer
> Affects Versions: 0.9.0.0
> Reporter: Rajini Sivaram
> Assignee: Rajini Sivaram
>
> Kafka producer gets metadata for topics when send is invoked and thereafter
> it attempts to keep the metadata up-to-date without any explicit requests
> from the client. This works well in static environments, but when topics are
> added or deleted, list of topics in Metadata grows but never shrinks. Apart
> from being a memory leak, this results in constant requests for metadata for
> deleted topics.
> We are running into this issue with the Confluent REST server where topic
> deletion from tests are filling up logs with warnings about unknown topics.
> Auto-create is turned off in our Kafka cluster.
> I am happy to provide a fix, but am not sure what the right fix is. Does it
> make sense to remove topics from the metadata list when
> UNKNOWN_TOPIC_OR_PARTITION response is received if there are no outstanding
> sends? It doesn't look very straightforward to do this, so any alternative
> suggestions are welcome.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
