Re: TaskManager job lifecycle hooks

2017-12-11 Thread Ben Sidhom
Ah, I see. Yes, that should definitely be good enough (at least for a first pass). Do you know how most users tend to deploy and use Flink in practice? Job-scoped clusters seem preferable for many reasons, but it's possible that that's only really practical when using YARN/Kubernetes/Mesos. Would

Re: TaskManager job lifecycle hooks

2017-12-10 Thread Aljoscha Krettek
Hi Ben, I think that's a good question but I also think that Erons answer is sufficient for an initial implementation. We suggest more and more to use a single "cluster" per job (either a single per-job YARN cluster or in Kubernetes or in Mesos), thus we don't really have to solve the problem o

Re: TaskManager job lifecycle hooks

2017-12-07 Thread Eron Wright
Could you speak to whether the lifecycle provided by RichFunction (open/close) would fit the requirement? https://ci.apache.org/projects/flink/flink-docs-release-1.3/api/java/org/apache/flink/api/common/functions/RichFunction.html#open-org.apache.flink.configuration.Configuration- On Thu, Dec 7,