Joao Boto created FLINK-13609:
---------------------------------
Summary: StreamingFileSink - reset part counter on bucket change
Key: FLINK-13609
URL: https://issues.apache.org/jira/browse/FLINK-13609
Project: Flink
Issue Type: Improvement
Components: Connectors / FileSystem
Reporter: Joao Boto
When writing to files using StreamingFileSink on bucket change we expect that
partcounter will reset its counter to 0
as a example
* using DateTimeBucketAssigner using ({color:#6a8759}yyyy/MM/dd/HH{color})
* and ten files hour (for simplicity)
this will create the:
* bucket 2019/08/07/00 with files partfile-0-0 to partfile-0-9
* bucket 2019/08/07/01 with files partfile-0-10 to partfile-0-19
* bucket 2019/08/07/02 with files partfile-0-20 to partfile-0-29
and we expect this:
* bucket 2019/08/07/00 with files partfile-0-0 to partfile-0-9
* bucket 2019/08/07/01 with files partfile-0-0 to partfile-0-9
* bucket 2019/08/07/02 with files partfile-0-0 to partfile-0-9
[~kkl0u] i don't know if it's the expected behavior (or this can be configured)
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
