Hi Arjun, I think this a great KIP and would be a great addition to have in connect. Had a couple of minor questions:
1. What would be the value in logging the connector config using errors.log.include.configs for every message? 2. Not being picky on format here but it might be clearer if the behavior is called out for each stage separately and what the connector developers need to do ( may be a tabular format). Also, I think all retriable exception when talking to Broker are never propagated to the Connect Framework since the producer is configured to try indefinitely 3. If a message fails in serialization, would the raw bytes be available to the dlq or the error log 4. Its not necessary to mention in KIP, but it might be better to separate the error records to a separate log file as part of the default log4j properties 5. If we disable message logging, would there be any other metadata available like offset that helps reference the record? 6. If I need error handler for all my connectors, would I have to set it up for each of them? I would think most people might want the behavior applied to all the connectors. Let me know your thoughts :). Thanks Magesh On Tue, May 8, 2018 at 11:59 PM, Arjun Satish <arjun.sat...@gmail.com> wrote: > All, > > I'd like to start a discussion on adding ways to handle and report record > processing errors in Connect. Please find a KIP here: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 298%3A+Error+Handling+in+Connect > > Any feedback will be highly appreciated. > > Thanks very much, > Arjun >
