Re: Gracefully stop long running streaming job

2016-04-26 Thread Maximilian Michels
I have to warn you that the Storm SpoutWrapper and the TwitterSource are currently the only stoppable sources. However, we could make more stoppable, e.g. the KafkaConsumer. On Tue, Apr 19, 2016 at 12:38 AM, Robert Schmidtke wrote: > I'm on 0.10.2 which seems to be still lacking this feature. Any

Re: Gracefully stop long running streaming job

2016-04-18 Thread Robert Schmidtke
I'm on 0.10.2 which seems to be still lacking this feature. Anyway I'm happy to see it'll be in future releases, so I'll get to enjoy it once I upgrade :) I'm using a FlinkKafkaConsumer081 for the record. Anyway, thanks a bunch Robert On Tue, Apr 19, 2016 at 12:14 AM, Matthias J. Sax wrote: > I

Re: Gracefully stop long running streaming job

2016-04-18 Thread Matthias J. Sax
If all your sources implements Stoppable interface, you can STOP a job. ./bin/flink stop JobID STOP is however quite new and it is ongoing work to make available sources stoppable (some are already). Not sure what kind of sources you are using right now. -Matthias On 04/18/2016 10:50 PM, Rober

Gracefully stop long running streaming job

2016-04-18 Thread Robert Schmidtke
Hi everyone, I am running a streaming benchmark which involves a potentially infinitely running Flink Streaming Job. I run it blocking on YARN using ./bin/flink run ... and then send the command into background, remembering its PID to kill it later on. While this gets the work done, the job always