Re: Flink complaining when trying to write to s3 in Parquet format

2020-02-12 Thread Kostas Kloudas
Hi Fatima, I am not super familiar with Parquet but your issue seems to be related to [1], which seems to be expected behaviour on the Parquet side. The reason for this behaviour seems to be the format of the parquet files which store only the leaf fields but not the structure of the groups, so if

Flink complaining when trying to write to s3 in Parquet format

2020-02-11 Thread Fatima Omer
I have a java app that is using a flink SQL query to perform aggregations on a data stream being read in from Kafka. Attached is the java file for reference. The query results are being written to s3. I can write successfully in Json format but when I try to use Parquet format, flink complains tha