I agree.
@Marton:
The idea with the extra thread does not work, because the method
JobClient.submitJobAndWait(...) does not return regularly if
ForkableFlinkMiniCluster.shutdown() is called -- instead an exception
occurs:
> Exception in thread "Thread-8" java.lang.RuntimeException:
> org.apache.
As a followup - I think it would be a good thing to add a way to gracefully
stop a streaming job.
Something that sends "close" to the sources, and they quit.
We can use this for graceful shutdown wen re-partitioninig / scaling in or
out, ...
On Wed, Apr 1, 2015 at 1:29 PM, Matthias J. Sax <
mj..
Hi,
I will pull the fix and try it out.
Thanks for the hint with the extra Thread. That should work for me. But
you are actually right; my setup is Storm inspired. I thinks its a very
natural way to deploy and stop and infinite streaming job. Maybe, you
want to adopt to it.
The ITCase I am writi
Hey Matthias,
Thanks for reporting the Exception thrown, we were not preparing for this
use case yet. We fixed it with Gyula, he is pushing a fix for it right now:
When the job is cancelled (for example due to shutting down the executor
underneath) you should not see that InterruptedException as s
Hi Robert,
thanks for your answer.
I get an InterruptedException when I call shutdown():
java.lang.InterruptedException
at java.lang.Object.wait(Native Method)
at java.lang.Thread.join(Thread.java:1225)
at java.lang.Thread.join(Thread.java:1278)
at
org.apache.flin
Hi Matthias,
the streaming folks can probably answer the questions better. But I'll
write something to bring this message back to their attention ;)
1) Which exceptions are you seeing? Flink should be able to cleanly shut
down.
2) As far as I saw it, the execute() method (of the Streaming API) go
Hi,
I am trying to run an infinite streaming job (ie, one that does not
terminate because it is generating output date randomly on the fly). I
kill this job with .stop() or .shutdown() method of
ForkableFlinkMiniCluster.
I did not find any example using a similar setup. In the provided
examples,
