Hi all, We encountered a similar situation in Debezium again, where an exception during Task::start() would be desirable to be retried.
Would anything speak against implementing retriable support for Task::start() in Kafka Connect? Would it require a KIP? Thanks, --Gunnar Am Mo., 9. Aug. 2021 um 10:47 Uhr schrieb Gunnar Morling < gunnar.morl...@googlemail.com>: > Hi, > > To ask slightly differently: would there be interest in a pull request for > implementing retries, in case RetriableException is thrown from the > Task::start() method? > > Thanks, > > --Gunnar > > > Am Do., 5. Aug. 2021 um 22:27 Uhr schrieb Sergei Morozov <moro...@tut.by>: > >> Hi, >> >> I'm trying to address an issue in Debezium (DBZ-3823 >> <https://issues.redhat.com/browse/DBZ-3823>) where a source connector >> task >> cannot recover from a retriable exception. >> >> The root cause is that the task interacts with the source database during >> SourceTask#start but Kafka Connect doesn't handle retriable exceptions >> thrown at this stage as retriable. KIP-298 >> < >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-298%3A+Error+Handling+in+Connect >> > >> that >> originally introduced handling of retriable exception doesn't describe >> handling task start exceptions, so it's unclear to me whether those aren't >> allowed by design or it was just out of the scope of the KIP. >> >> My current working solution >> <https://github.com/debezium/debezium/pull/2572> relies >> on the internal Debezium implementation of the task restart which >> introduces certain risks (the details are in the PR description). >> >> The question is: are retriable exceptions during start disallowed by >> design, and the task must not throw retriable exceptions during start, or >> it's just currently not supported by the Connect framework and I just need >> to implement proper error handling in the connector? >> >> Thanks! >> >> -- >> Sergei Morozov >> >
