Hello all, In multiple jobs I'm saving data using the datastream API with StreamingFileSink and various bulk formats (avro, parquet). As bulk formats require a rolling policy that extends the CheckpointRollingPolicy I have created a policy that rolls on file size additionally. Unfortunately for some flows the data velocity is not that big and even with 128MB max file size and 6 hour checkpoint interval I end up with small files.
I have seen that there are some open tickets for this, but my question is if there are any solutions/workarounds for this in flink 1.13.2? Best regards Kamil