[
https://issues.apache.org/jira/browse/FLINK-33783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17806001#comment-17806001
]
Bartosz Mikulski commented on FLINK-33783:
------------------------------------------
Sure, sounds great [~martijnvisser] !
It would be great if I could create a FLIP, thank you! Any tips on how to write
one properly?
How can I open a mailing list discussion, is it from a FLIP page?
> Add options to ignore parsing error in Kafka SQL Connector
> ----------------------------------------------------------
>
> Key: FLINK-33783
> URL: https://issues.apache.org/jira/browse/FLINK-33783
> Project: Flink
> Issue Type: Improvement
> Components: Connectors / Kafka, Table SQL / API
> Reporter: Bartosz Mikulski
> Priority: Major
>
> h1. Current state
> If an unparsable event enters a Flink Kafka Source in SQL then the whole
> application restarts. For JSON format there is a property that allows to
> ignore unparsable events. Other formats, like `confluent-avro` etc don't
> support that.
> h1. Desired state
> We would like to ignore the parsing exception in Kafka Source in SQL
> regardless of the format used. Additionally, a new metric should be
> introduced that returns a count of unparsable events seen so far.
> In the future there should be a Dead Letter Queue handling in SQL Sources
> similar to Kafka Streams:
> [https://www.confluent.io/blog/kafka-connect-deep-dive-error-handling-dead-letter-queues/].
> For now, the universal ignore with metric would be enough.
> h1. Implementation
> We already have an implementation for this case in the Flink Kafka Connector
> and we would like to open a pull request for it. However, we created the
> issue first as per this recommendation
> [https://flink.apache.org/how-to-contribute/contribute-code/]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
