Re: [DISCUSS] Future of Per-Job Mode

2022-01-24 Thread Matthias Pohl
Hi all, I agree with Xintong's comment: Reducing the number of deployment modes would help users. There is a clearer distinction between session mode and the two other deployment modes (i.e. application and job mode). The difference between application and job mode is not that easy to grasp for new

Re: [DISCUSS] Future of Per-Job Mode

2022-01-24 Thread Xintong Song
Sorry for joining the discussion late. I'm leaning towards deprecating the per-job mode soonish, and eventually dropping it in the long-term. - One less deployment mode makes it easier for users (especially newcomers) to understand. Deprecating the per-job mode sends the signal that it is legacy,

Re: [DISCUSS] Future of Per-Job Mode

2022-01-21 Thread Konstantin Knauf
Thanks Thomas & Biao for your feedback. Any additional opinions on how we should proceed with per job-mode? As you might have guessed, I am leaning towards proposing to deprecate per-job mode. On Thu, Jan 13, 2022 at 5:11 PM Thomas Weise wrote: > Regarding session mode: > > ## Session Mode > *

Re: [DISCUSS] Future of Per-Job Mode

2022-01-13 Thread Thomas Weise
Regarding session mode: ## Session Mode * main() method executed in client Session mode also supports execution of the main method on Jobmanager with submission through REST API. That's how Flinkk k8s operators like [1] work. It's actually an important capability because it allows for allocation

Re: [DISCUSS] Future of Per-Job Mode

2022-01-13 Thread Biao Geng
Hi Konstantin, Thanks a lot for starting this discussion! I hope my thoughts and experiences why users use Per-Job Mode, especially in YARN can help: #1. Per-job mode makes managing dependencies easier: I have met some customers who used Per-Job Mode to submit jobs with a lot of local user-defined

[DISCUSS] Future of Per-Job Mode

2022-01-13 Thread Konstantin Knauf
Hi everyone, I would like to discuss and understand if the benefits of having Per-Job Mode in Apache Flink outweigh its drawbacks. *# Background: Flink's Deployment Modes* Flink currently has three deployment modes. They differ in the following dimensions: * main() method executed on Jobmanager