[
https://issues.apache.org/jira/browse/KAFKA-14015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17585390#comment-17585390
]
Yash Mayya edited comment on KAFKA-14015 at 8/27/22 8:51 AM:
-------------------------------------------------------------
In essence, this should work as expected when Connect is being run in
distributed mode. In standalone mode, it looks like there is a similar flow
[here|https://github.com/apache/kafka/blob/38103ffaa962ef5092baffb884c84f8de3568501/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerder.java#L392-L407]
(i.e. get latest connector configs -> ask connector for task configs -> check
if there's a change in the task configs -> if so, stop old set of tasks and
start new set of tasks). However, that seems to only be called if i) configs
are updated directly via PUT /config endpoint; ii) connector explicitly
requests task reconfiguration via its context; iii) connector is resumed via
the REST API. So we might need to make some changes to ensure that task
reconfiguration is requested when
[restartConnector|https://github.com/apache/kafka/blob/38103ffaa962ef5092baffb884c84f8de3568501/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerder.java#L287]
is called in the StandaloneHerder (might be as simple as adding a call to
[updateConnectorTasks|https://github.com/apache/kafka/blob/38103ffaa962ef5092baffb884c84f8de3568501/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerder.java#L392-L407]
in startConnector)
was (Author: yash.mayya):
In essence, this should work as expected when Connect is being run in
distributed mode. In standalone mode, it looks like there is a similar flow
[here|https://github.com/apache/kafka/blob/38103ffaa962ef5092baffb884c84f8de3568501/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerder.java#L392-L407]
(i.e. get latest connector configs -> ask connector for task configs -> check
if there's a change in the task configs -> if so, stop old set of tasks and
start new set of tasks). However, that seems to only be called if i) configs
are updated directly via PUT /config endpoint; ii) connector explicitly
requests task reconfiguration via its context; iii) connector is resumed via
the REST API. So we might need to make some changes to ensure that task
reconfiguration is requested when
[restartConnector|https://github.com/apache/kafka/blob/38103ffaa962ef5092baffb884c84f8de3568501/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerder.java#L287]
is called in the StandaloneHerder (might be as simple as adding a call to
[updateConnectorTasks|https://github.com/apache/kafka/blob/38103ffaa962ef5092baffb884c84f8de3568501/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerder.java#L392-L407]
in restartConnector)
> ConfigProvider with ttl fails to restart tasks
> ----------------------------------------------
>
> Key: KAFKA-14015
> URL: https://issues.apache.org/jira/browse/KAFKA-14015
> Project: Kafka
> Issue Type: Bug
> Reporter: Ross Lawley
> Assignee: Sagar Rao
> Priority: Major
>
> According to the
> [KIP-297|https://cwiki.apache.org/confluence/display/KAFKA/KIP-297%3A+Externalizing+Secrets+for+Connect+Configurations#KIP297:ExternalizingSecretsforConnectConfigurations-SecretRotation]:
> {quote} * When the Herder receives the onChange() call, it will check a new
> connector configuration property config.reload.action which can be one of the
> following:
> ** The value restart, which means to schedule a restart of the Connector and
> all its Tasks. This will be the default.
> ** The value none, which means to do nothing.{quote}
> However, the
> [restartConnector|https://github.com/apache/kafka/blob/3.2.0/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/standalone/StandaloneHerder.java#L287-L294]
> method only restarts the connector and does not restart any tasks. Suggest
> calling {{restartConnectorAndTasks}} instead.
> The result is changed configurations provided by the ConfigProvider are not
> picked up and existing tasks continue to use outdated configuration.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
