Hello,
we have a requirement to make kafka connector more fault tolerant for our
use, where in we don't want them to fail for some kinds of errors, i.e
error where bootstrap broker are missing or if we don't have sufficient
permission to read data from topic (we are reading from aws msk).

we tried using basic error handing for such exception where in, in the poll
method we tried to swallow SaslException and config exceptions where
bootstrap brokers are missing, and retry after t amount of time but this
seems to make the connector make lots of unassigned task corresponding to
even a connector even with max.tasks set to 1.

1. unclear as to why that happens
2. looking for guidance on more standardised way to ensure resilient
connectors that don't transition to fail state on some errors which we
expect can happen

Reply via email to