Re: Streaming File Sink - Parquet File Writer

Hi Vinay, You are correct when saying that the bulk formats only support onCheckpointRollingPolicy. The reason for this has to do with the fact that currently Flink relies on the Hadoop writer for Parquet. Bulk formats keep important details about how they write the actual data (such as compress

Streaming File Sink - Parquet File Writer

Hi, I am not able to roll the files based on file size as the bulkFormat has onCheckpointRollingPolicy. One way is to write CustomStreamingFileSink and provide RollingPolicy like RowFormatBuilder. Is this the correct way to go ahead ? Another way is to write ParquetEncoder and use RowFormatBuild