Hi, I would like to propose an improvement that would enable reading table columns from different parts of source records. Besides the main payload majority (if not all of the sources) expose additional information. It can be simply a read-only metadata such as offset, ingestion time or a read and write parts of the record that contain data but additionally serve different purposes (partitioning, compaction etc.), e.g. key or timestamp in Kafka.
We should make it possible to read and write data from all of those locations. In this proposal I discuss reading partitioning data, for completeness this proposal discusses also the partitioning when writing data out. I am looking forward to your comments. You can access the FLIP here: https://cwiki.apache.org/confluence/display/FLINK/FLIP-107%3A+Reading+table+columns+from+different+parts+of+source+records?src=contextnavpagetreemode Best, Dawid
signature.asc
Description: OpenPGP digital signature
