lihe ma created FLINK-22918:
-------------------------------
Summary: StreamingFileSink does not commit partition when no
message is sent between checkpoints
Key: FLINK-22918
URL: https://issues.apache.org/jira/browse/FLINK-22918
Project: Flink
Issue Type: Improvement
Components: Connectors / FileSystem
Affects Versions: 1.11.3
Reporter: lihe ma
FileSink extract the partitions from messages , then send partition list to
committer. In dynamic partition case, if one partition contains no data
between two checkpoints. then the partition wont commit.
like this:
date=0101/hour=01/key=a/part1-file (write at 01-01 01:20:00)
if there is no data with key=a between 01:20 and 02:10 (a time when watermark
passed) , the committer will not receive this partition and it will never
commit.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
