Piotr Rybarczyk created KAFKA-12990:
---------------------------------------
Summary: Infinite Loop in producing event to kafka
Key: KAFKA-12990
URL: https://issues.apache.org/jira/browse/KAFKA-12990
Project: Kafka
Issue Type: Bug
Environment: Kafka 2.4.1
Kafka Connect 5.4.3
Reporter: Piotr Rybarczyk
Marked as a bug, but depends on what configuration means.
We are using AWS RDS as a Postgres database based on Postgres 11 version.
We are using *Debezium 1.4.1* version with *pgoutput* and *outbox router.*
Recently, I've been testing what will happen if the topic will not be
configured in Kafka.
Normally, in that case, connector login warns & stuck in an infinite loop as
the task is unable to produce an event to Kafka:
{code:java}
WARN [Producer
clientId=connector-producer-piotrek-test-3.outbox_events_debezium_connector-0]
Error while fetching metadata with correlation id 3884 :
{piotrek-test-123.partners.feature-flag-events=UNKNOWN_TOPIC_OR_PARTITION}
(org.apache.kafka.clients.NetworkClient)
{code}
I've confirmed with the Debezium team that this issue is not in the connector
logic scope. From my guess, it needs to be on Kafka or Kafka Connect level.
The worst thing is that even when I've disabled the connector, & task logs
still have been produced to console no matter what I've done. So in that case,
we can not really on task statuses to check is pipeline working, we need to
verify it on logs level.
More info in
- [Debezium Issue|https://issues.redhat.com/browse/DBZ-3652]
-
[Slack|https://confluentcommunity.slack.com/archives/C49L0V3L7/p1624448948171100]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
