Hi Akash, if your connector doesn't have the appropriate permissions for a topic then it can't run - so I'm not sure what value there is in trying to handle or tolerate that kind of exception. If someone is changing ACLs in such a way that it breaks a connector, then you probably want that connector to go into a failed state. I don't think there's a way to change your connector code to handle this since consuming/producing to topics is handled by the Connect framework, but in my opinion if people are repeatedly breaking ACLs, then that might be better fixed with a process change? (if this is an ongoing problem then it sounds like there's a larger issue happening that should be addressed)
When you say "bootstrap brokers are missing"... are you referring to someone misconfiguring a connector such that the bootstrap servers config is missing? - alex On Wed, Jun 12, 2024 at 4:16 PM Akash Dhiman <akashdhiman...@gmail.com> wrote: > Hello, > we have a requirement to make kafka connector more fault tolerant for our > use, where in we don't want them to fail for some kinds of errors, i.e > error where bootstrap broker are missing or if we don't have sufficient > permission to read data from topic (we are reading from aws msk). > > we tried using basic error handing for such exception where in, in the poll > method we tried to swallow SaslException and config exceptions where > bootstrap brokers are missing, and retry after t amount of time but this > seems to make the connector make lots of unassigned task corresponding to > even a connector even with max.tasks set to 1. > > 1. unclear as to why that happens > 2. looking for guidance on more standardised way to ensure resilient > connectors that don't transition to fail state on some errors which we > expect can happen >
