[
https://issues.apache.org/jira/browse/FLINK-15526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17361380#comment-17361380
]
luoyuxia edited comment on FLINK-15526 at 6/11/21, 3:59 AM:
------------------------------------------------------------
[~lzljs3620320] I see DefaultRollingPolicy.shouldRollOnEvent can already avoid
writing a big file by file size. Does the Jira aim to add a policy that can
avoid writing a big file by record numebr?
was (Author: luoyuxia):
[~lzljs3620320] I see DefaultRollingPolicy.shouldRollOnEvent already can avoid
writing a big file by file size. Does the Jira aim to add a policy that can
avoid writing a big file by record numebr?
> Add RollingPolicy to PartitionWriters
> -------------------------------------
>
> Key: FLINK-15526
> URL: https://issues.apache.org/jira/browse/FLINK-15526
> Project: Flink
> Issue Type: Sub-task
> Components: Table SQL / Ecosystem
> Reporter: Jingsong Lee
> Priority: Major
>
> Now our partition writers will write a whole file in a checkpoint if there is
> no partition change.
> Sometimes this file is too big, it is better to add a RollingPolicy for
> controlling file size.
> We can just add policy controlled by file size and record number.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
