remove
>>> from the state the files that it *really successfully* sinks (perhaps
>>> you may want to add a validation here with S3 to check file integrity).
>>> This leaves us in the state with the failed files, partial files etc.
>>>
>>>
>>>
kes
> s3 consistent and Streaming file sink works just fine.
>
>
>
> On Sat, Aug 17, 2019, 3:32 AM Swapnil Kumar wrote:
>
>> Hello, We are using Flink to process input events and aggregate and write
>> o/p of our streaming job to S3 using StreamingFileSink but whenev
Hello Rafi,
Thank you for getting back. We have lifecycle rule setup for the sink and
not the s3 bucket for savepoints. This was my initial hunch too but we
tried restarting the job immediately after canceling them and it failed.
Best,
Swapnil Kumar
On Sat, Aug 17, 2019 at 2:23 PM Rafi Aroch
) files and can no longer be found on s3. Is there a
workaround for this, so that we can use s3 as a sink?
--
Thanks,
Swapnil Kumar
