Re: TaskManager job lifecycle hooks

Ah, I see. Yes, that should definitely be good enough (at least for a first pass). Do you know how most users tend to deploy and use Flink in practice? Job-scoped clusters seem preferable for many reasons, but it's possible that that's only really practical when using YARN/Kubernetes/Mesos. Would

Re: TaskManager job lifecycle hooks

Hi Ben, I think that's a good question but I also think that Erons answer is sufficient for an initial implementation. We suggest more and more to use a single "cluster" per job (either a single per-job YARN cluster or in Kubernetes or in Mesos), thus we don't really have to solve the problem o

TaskManager job lifecycle hooks

(Continuing previous thread because I made the mistake of subscribing to the digest rather than direct emails.) Thanks, Eron. RichFunction seems to be a step in the right direction (the docs imply that the lifecycle is per set of tasks on a TaskManager per high-level function rather than per task)

Re: TaskManager job lifecycle hooks

Could you speak to whether the lifecycle provided by RichFunction (open/close) would fit the requirement? https://ci.apache.org/projects/flink/flink-docs-release-1.3/api/java/org/apache/flink/api/common/functions/RichFunction.html#open-org.apache.flink.configuration.Configuration- On Thu, Dec 7,

TaskManager job lifecycle hooks

Hey, I'm working on the Apache Beam portability story and trying to figure out how we can get the Flink runner to support the new portability API . In order to get the runner to work with portable SDKs, we need to be able