Re: Flink job cannot find recover path after using entropy injection for s3 file systems

2021-03-19 Thread Rainie Li
ropy functionality. > > My suspicion is that with FLINK-5763 (this has been introduced with Flink > 1.11) we moved the metadata file also under the entropy folder to make the > checkpoints/savepoints self-contained and relocatable. > > Cheers, > Till > > On Wed, Mar 17, 2

Re: Flink job cannot find recover path after using entropy injection for s3 file systems

2021-03-17 Thread Rainie Li
s.apache.org/jira/browse/FLINK-5763 > [3] https://issues.apache.org/jira/browse/FLINK-9061 > > Cheers, > Till > > On Tue, Mar 16, 2021 at 7:03 PM Rainie Li > wrote: > > > Hi Flink Developers. > > > > We enabled entropy injection for s3, here is our se

Flink job cannot find recover path after using entropy injection for s3 file systems

2021-03-16 Thread Rainie Li
Hi Flink Developers. We enabled entropy injection for s3, here is our setting on Yarn Cluster. s3.entropy.key: _entropy_ s3.entropy.length: 1 state.checkpoints.dir: 's3a://{bucket name}/dev/checkpoints/_entropy_' I have two questions: 1. After enabling entropy, job's checkpoint path changed to: *