Re: Exception when trying to change StreamingFileSink S3 bucket

2019-09-05 Thread Kostas Kloudas
Hi Sidhartha, Your explanation is correct. If you stopped the job with a savepoint and then you try to restore from that savepoint, then Flink will try to restore its state which is, of course, included in its old bucket. But new data will go to the new bucket. One solution is either to restart

Re: Exception when trying to change StreamingFileSink S3 bucket

2019-09-05 Thread Fabian Hueske
Hi, Kostas (in CC) might be able to help. Best, Fabian Am Mi., 4. Sept. 2019 um 22:59 Uhr schrieb sidhartha saurav < sidsau...@gmail.com>: > Hi, > > Can someone suggest a workaround so that we do not get this issue while > changing the S3 bucket ? > > On Thu, Aug 22, 2019 at 4:24 PM sidhartha s

Re: Exception when trying to change StreamingFileSink S3 bucket

2019-09-04 Thread sidhartha saurav
Hi, Can someone suggest a workaround so that we do not get this issue while changing the S3 bucket ? On Thu, Aug 22, 2019 at 4:24 PM sidhartha saurav wrote: > Hi, > > We are trying to change our StreamingFileSink S3 bucket, say from s3:// > *eu1/output_old* to s3://*eu2/output_new*. When we do

Exception when trying to change StreamingFileSink S3 bucket

2019-08-22 Thread sidhartha saurav
Hi, We are trying to change our StreamingFileSink S3 bucket, say from s3:// *eu1/output_old* to s3://*eu2/output_new*. When we do so we get an exception and the taskmanger goes into a restart loop. We suspect that it tries to restore state and gets the bucketid from saved state [* final BucketID