[
https://issues.apache.org/jira/browse/KAFKA-12990?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17507303#comment-17507303
]
Chris Egerton commented on KAFKA-12990:
---------------------------------------
Worth noting that with [https://github.com/apache/kafka/pull/10016], which has
been backported back through the 2.5 branch, tasks caught in this loop should
be interrupted when scheduled for shutdown. This prevents Kafka Connect from
accumulating multiple producers for a single task that has been restarted
multiple times (due to reconfiguration, rebalance, manual request via the REST
API, etc.), but doesn't prevent tasks from getting blocked like this in the
first place when writing to nonexistent topics, and doesn't provide better
insight into how/when tasks enter that state.
> Infinite Loop in producing event to kafka
> -----------------------------------------
>
> Key: KAFKA-12990
> URL: https://issues.apache.org/jira/browse/KAFKA-12990
> Project: Kafka
> Issue Type: Bug
> Environment: Kafka 2.4.1
> Kafka Connect 5.4.3
> Reporter: Piotr Rybarczyk
> Priority: Major
>
> Marked as a bug, but depends on what configuration means.
> We are using AWS RDS as a Postgres database based on Postgres 11 version.
> We are using *Debezium 1.4.1* version with *pgoutput* and *outbox router.*
> Recently, I've been testing what will happen if the topic will not be
> configured in Kafka.
> Normally, in that case, connector login warns & stuck in an infinite loop as
> the task is unable to produce an event to Kafka:
> {code:java}
> WARN [Producer
> clientId=connector-producer-piotrek-test-3.outbox_events_debezium_connector-0]
> Error while fetching metadata with correlation id 3884 :
> {piotrek-test-123.partners.feature-flag-events=UNKNOWN_TOPIC_OR_PARTITION}
> (org.apache.kafka.clients.NetworkClient)
> {code}
> I've confirmed with the Debezium team that this issue is not in the connector
> logic scope. From my guess, it needs to be on Kafka or Kafka Connect level.
> The worst thing is that even when I've disabled the connector, & task logs
> still have been produced to console no matter what I've done. So in that
> case, we can not really on task statuses to check is pipeline working, we
> need to verify it on logs level.
> More info in
> - [Debezium Issue|https://issues.redhat.com/browse/DBZ-3652]
> -
> [Slack|https://confluentcommunity.slack.com/archives/C49L0V3L7/p1624448948171100]
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
