Hi devs, When working on improving the Table API/SQL connectors we faced a few shortcomings of the DeserializationSchema and SerializationSchema interfaces. Similar features were also mentioned by other users in the past. The shortcomings I would like to address with the FLIP include:
* Emitting 0 to m records from the deserialization schema with per
partition watermarks
o https://github.com/apache/flink/pull/3314#issuecomment-376237266
o differentiate null value from no value
o support for Debezium CDC format
(https://cwiki.apache.org/confluence/display/FLINK/FLIP-105%3A+Support+to+Interpret+and+Emit+Changelog+in+Flink+SQL)
* A way to initialize the schema
o establish external connections
o generate code on startup
o no need for lazy initialization
* Access to metrics
[http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Custom-Metrics-outside-RichFunctions-td32282.html#a32329]
One important aspect I would like to hear your opinion on is how to
support the Collector interface in Kafka source. Of course if we agree
to add the Collector to the DeserializationSchema.
The FLIP can be found here:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=148645988&src=contextnavpagetreemode
Looking forward to your feedback.
Best,
Dawid
signature.asc
Description: OpenPGP digital signature
