Hi Yang, Thanks for your quick reply.
The Flink K8s documentation distinguishes between standalone and session deployment mode. I think I will use the latter. Since my previous mail, I found FLIP-53 <https://cwiki.apache.org/confluence/display/FLINK/FLIP-53%3A+Fine+Grained+Operator+Resource+Management> which is precisely the topic of my original question. So, great progress has been already made to cover my needs. Unfortunately, I am use DataStreams API which are currently not covered by the initial implementation. I have asked on the dev mailing list if I could help bridging this gap. Regards, Michaël Le jeu. 19 déc. 2019 à 04:58, Yang Wang <danrtsey...@gmail.com> a écrit : > Hi Michaël, > > Glad to hear that you are going to run Flink workload on Kubernetes. > AFAIK, we have two > deployment ways. > 1. Running Flink standalone session/per-job cluster on K8s. You need to > calculate how many > taskmanagers you need and the <memory, cpu> per taskmanager. All the > taskmanager > will be started by a K8s deployment. You could find more information > here[1]. In this mode, > you could be `kubectl scale` to change the replicas of taskmanager if the > resources are not > enough for your job. > 2. Natively running Flink session/per-job on K8s. The session mode has > been support in > master branch and will be released in 1.10. The per-job mode is in > discussion. No matter > session or per-job, the taskmanager will be allocated dynamically on > demand. You could > use a simple command to start a Flink cluster on K8s. More information > could be found > here[2]. > > > Best, > Yang > > [1]. > https://ci.apache.org/projects/flink/flink-docs-master/ops/deployment/kubernetes.html > [2]. > https://docs.google.com/document/d/1-jNzqGF6NfZuwVaFICoFQ5HFFXzF5NVIagUZByFMfBY/edit?usp=sharing > > > Michaël Melchiore <rohe...@gmail.com> 于2019年12月19日周四 上午1:11写道： > >> Hello, >> >> I plan to run topologies on a Flink session cluster on Kubernetes. >> In my topologies, operators will have varying resource requirements in >> term of CPU and RAM. >> How can I make these informations available from Flink to Kubernetes so >> the latter takes it into account to optimize its deployment ? >> >> I am trying to achieve something similar to Apache Storm/Trident Resource >> Aware Scheduler >> <https://storm.apache.org/releases/2.0.0/Trident-RAS-API.html>. >> >> Kind regards, >> >> Michaël >> >
