Correctly implementing of SourceFunction.run()
Hey Kumar,
if you are swallowing any and all exceptions, your Flink job will not fail
because of issues arising from your custom source. It might make sense to stop
the source if you are catching an InterruptedException.
Throwing exceptions out of the
Hey Kumar,
if you are swallowing any and all exceptions, your Flink job will not fail
because of issues arising from your custom source. It might make sense to
stop the source if you are catching an InterruptedException.
Throwing exceptions out of the run method basically signals the Flink
framew
Hi,
Some suggestions from my side:
- synchronized (checkpointLock) to some work and ctx.collect?
- Put Thread.sleep(interval) out of try catch? Maybe should not
swallow interrupt exception (Like cancel the job).
Best,
Jingsong Lee
On Fri, May 8, 2020 at 2:52 AM Senthil Kumar wrote:
> I am impl
I am implementing a source function which periodically wakes up and consumes
data from S3.
My currently implementation is like so.
Following:
org.apache.flink.streaming.api.functions.source.ContinuousFileMonitoringFunction
Is it safe to simply swallow any and all exceptions in the run method
