Re: Test sink behaviour

2020-02-17 Thread Till Rohrmann
Hi David, if you want to test the behavior together with S3, then you could check that S3 contains a file after the job has completed. If you want to test the failure and retry behaviour, then I would suggest to introduce an own abstraction for the S3 access which you can control. That way you ca

Test sink behaviour

2020-02-13 Thread David Magalhães
Hi, I've created a CustomSink that writes parquet file to S3. Inside the `invoke` method I have a loop to check if S3 is down, and if it is it will wait exponentially until it is online again. Now I want to write a test for this, and I can execute everything and see that the Sink is doing what is