Seth Wiesman created FLINK-11499:
------------------------------------
Summary: Extend StreamingFileSink BulkFormats to support arbitrary
roll policy's
Key: FLINK-11499
URL: https://issues.apache.org/jira/browse/FLINK-11499
Project: Flink
Issue Type: Improvement
Reporter: Seth Wiesman
Currently when using the StreamingFilleSink Bulk-encoding formats can only be
combined with the `OnCheckpointRollingPolicy`, which rolls the in-progress part
file on every checkpoint.
However, many bulk formats such as parquet are most efficient when written as
large files; this is not possible when frequent checkpointing is enabled.
Currently the only work-around is to have long checkpoint intervals which is
not ideal.
The StreamingFileSink should be enhanced to support arbitrary roll policy's so
users may write large bulk files while retaining frequent checkpoints.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
