Re: Pyflink/Flink Java parquet streaming file sink for a dynamic schema stream

2021-12-03 Thread Georg Heiler
Hi, the schema of the after part depends on each table i.e. holds different columns for each table. So do you receive debezium changelog statements for all/ >1 table? I.e. is the schema in the after part different? Best, Georg Am Fr., 3. Dez. 2021 um 08:35 Uhr schrieb Kamil ty : > Yes the gener

Re: Pyflink/Flink Java parquet streaming file sink for a dynamic schema stream

2021-12-02 Thread Georg Heiler
Do the JSONs have the same schema overall? Or is each potentially structured differently? Best, Georg Am Fr., 3. Dez. 2021 um 00:12 Uhr schrieb Kamil ty : > Hello, > > I'm wondering if there is a possibility to create a parquet streaming file > sink in Pyflink (in Table API) or in Java Flink (in

Pyflink/Flink Java parquet streaming file sink for a dynamic schema stream

2021-12-02 Thread Kamil ty
Hello, I'm wondering if there is a possibility to create a parquet streaming file sink in Pyflink (in Table API) or in Java Flink (in Datastream api). To give an example of the expected behaviour. Each element of the stream is going to contain a json string. I want to save this stream to parquet