[
https://issues.apache.org/jira/browse/KAFKA-15041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17727894#comment-17727894
]
Sagar Rao commented on KAFKA-15041:
-----------------------------------
Ideally a worker restart should fix this but it didn't quite happen on my local
testing.
> Source Connector auto topic creation fails when topic is deleted and brokers
> don't support auto topic creation
> --------------------------------------------------------------------------------------------------------------
>
> Key: KAFKA-15041
> URL: https://issues.apache.org/jira/browse/KAFKA-15041
> Project: Kafka
> Issue Type: Bug
> Components: KafkaConnect
> Reporter: Sagar Rao
> Assignee: Sagar Rao
> Priority: Major
>
> [KIP-158|https://cwiki.apache.org/confluence/display/KAFKA/KIP-158%3A+Kafka+Connect+should+allow+source+connectors+to+set+topic-specific+settings+for+new+topics]
> allows the source connectors to create topics even when the broker doesn't
> allow to do so. It does so by checking for every record if a topic needs to
> be created
> [https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/AbstractWorkerSourceTask.java#L500.]
> To not always keep checking for topic presence via admin topics, it also
> maintains a cache of the topics that it has created and doesn't create those
> anymore. This helps to create topics when brokers don't support automatic
> topic creation.
> However, lets say the topic gets created initially and later on gets deleted
> while the connector is still running and the brokers don't support automatic
> topic creation. For such cases, the connector has cached the topic it has
> already created and wouldn't recreate it because the cache never updates and
> since the broker doesn't support topic creation, the logs would just be full
> of messages like
> ```
> Error while fetching metadata with correlation id 3260 :
> \{connect-test=UNKNOWN_TOPIC_OR_PARTITION}
> ```
>
> This can become a problem on enviroments where brokers don't allow topic
> creation. We need a way to refresh the topics cache for such cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
