Re: What's the purpose of uniqueId in FileWriterBucket

2021-10-10 Thread Yun Gao
4 smoothly? From: Yun Gao Date: Friday, October 8, 2021 at 22:43 To: wu shaoj , dev@flink.apache.org Subject: Re: What's the purpose of uniqueId in FileWriterBucket Hi Wu, The uid is used to distinguish between the different subtasks, if removed, the different subtasks of the filesink

Re: What's the purpose of uniqueId in FileWriterBucket

2021-10-08 Thread wu shaoj
If flink job recover from previous checkpoint/savepoint, will it re-output records to a different file with same partFileIndex? And can we upgrade flink to 1.4 smoothly? From: Yun Gao Date: Friday, October 8, 2021 at 22:43 To: wu shaoj , dev@flink.apache.org Subject: Re: What's the pu

Re: What's the purpose of uniqueId in FileWriterBucket

2021-10-08 Thread Yun Gao
Hi Wu, The uid is used to distinguish between the different subtasks, if removed, the different subtasks of the filesink would have name conflicts if they writes to the same bucket, thus the uid should be necessary if there are multiple subtasks. Best, Yun --Original Mail --