Hi Ivan,
For flink sink it might commit a single file multiple times. This happens if
there are failover after committing one file and then met with a failover, the
the job will restarted from the latest checkpoint, and the file's state will be
get back to pending and committed again. In the
Hi Yun,
Thank you so much for you suggestion.
(1) The job couldn’t restore from the last checkpoint. The exception is in my
original email.
(2) No, I didn’t change any multipart upload settings.
(3) The file is gone. I have another batch process that reads Flink output s3
bucket and pushes obj
Hi Ivan,
I think there might be some points to check:
1. Is the job restored from the latest successful checkpoint after restart ?
2. Have you ever changed the timeout settings for uncompleted multipart
upload ?
3. Does cbd/late-landing/event_date=2020-08-28/event_hour=16/part-5-26
