Re: Caveats of multi-execute() Flink programs

2020-01-23 Thread Kostas Kloudas
Hi all, I agree with Stephan that it is reasonable to assume that multiple independent jobs should not be in the same "main()" or "application" or "program" or environment. But if/when dataStream subsumes dataSet, then batch programs will also have checkpoints/savepoints and in that case it is n

Re: Caveats of multi-execute() Flink programs

2020-01-23 Thread Aljoscha Krettek
@Stephan: yes, I agree about how embedded (and driver) mode should work. I have a PoC from last year that does just that: https://github.com/aljoscha/flink/commit/ba0931cd307c5e7e68e903318e116ed550bb20eb#diff-7752e46061b390a7a71328dac4be54ff But that doesn't solve the weirdness around specifyin