If delete expired topic, kafka consumer will keep flushing unknown_topic warning in log

2019-03-21 Thread Shengnan YU
The issue has been created in JIRA https://issues.apache.org/jira/browse/KAFKA-8100. But no one has replied me yet. Recently we used flink to consume kafka topics with a regex pattern. It is found that when we deleted some unused topics, the logs will keep flushing UNKNOWN_TOPIC_EXCEPTION. I

Kafka client keeps fetch metadata for deleted topic, which leads to UNKNOWN_TOPIC_EXCEPTION

2019-04-14 Thread Shengnan YU
Recently we used Apache Flink to consume kafka topics with a regex pattern. It is found that when we deleted some unused topics, the logs will keep flushing UNKNOWN_TOPIC_EXCEPTION. I looked up the source code of kafka client and found that for consumer, topicExpiry is disable in Metadata, whic

kafka consumer metadata expire

2019-04-23 Thread Shengnan YU
Hi everyone How to update kafka consumer's metadata when some topics are deleted. The metadata in consumer will not be expired or remove outdated topic, which leads to UNKNOWN_TOPIC exception when fetching metadata from the cluster. Thank you very mych.