[
https://issues.apache.org/jira/browse/KAFKA-14746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17707886#comment-17707886
]
Mickael Maison commented on KAFKA-14746:
----------------------------------------
To be honest I'm not sure if we should change this behavior or simply document
it. I tend to agree that the retry logic was likely intended to handle failures
communicating with the leader instead of exceptions from taskConfigs().
> Throwing in Connector.taskConfigs in distributed mode generates a lot of logs
> -----------------------------------------------------------------------------
>
> Key: KAFKA-14746
> URL: https://issues.apache.org/jira/browse/KAFKA-14746
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Mickael Maison
> Priority: Major
>
> If a Connector throws in its taskConfigs() method, the runtime ends up
> retrying using DistributedHerder.RECONFIGURE_CONNECTOR_TASKS_BACKOFF_MS which
> is a fixed value (250ms). For each retry, the runtime prints the connector
> configuration and the enriched configuration so this can quickly generate a
> lot of logs.
> There is some value in throwing in taskConfigs() as it allows to fail fast in
> case the connector is given bad credentials. For example this is what some of
> the Debezium connectors do:
> https://github.com/debezium/debezium/blob/main/debezium-connector-sqlserver/src/main/java/io/debezium/connector/sqlserver/SqlServerConnector.java#L56-L69
> The way Connectors are expected to work today is to instead always create
> tasks and let each task fail in case the configuration is wrong. We should
> document that and make it clear in the javadoc that throwing in taskConfigs
> is not recommended.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
