Re: Elastic scaling with HA (zookeeper)

2023-05-06 Thread Weihua Hu
Hi, If I provide the same jobId, will it restore from the HA first, or just > start anew? it will restore from HA first, and will not start a new one. I tried with flink v1.15.2 with the flowing example, but can't reproduce. Could you provide the full start command? 1. ./bin/standalone-job.sh

Re: Elastic scaling with HA (zookeeper)

2023-05-06 Thread Shubham Bansal
Thanks for the response Weihua. I am using flink v.15.2. I am unable to understand how I should be starting the jobmanager without the job so that it can poll HA store and start the job. Since the current method of running the script standalone-job.sh requires a job at start. If I provide the same

Re: Elastic scaling with HA (zookeeper)

2023-05-05 Thread Weihua Hu
Hi, Shubham Which Flink version are you using? AFAIK, the JobManager will recover the job from the HA store first, and it won't submit the same job (identify be jobID) again if it has already been recovered. Best, Weihua On Tue, May 2, 2023 at 8:02 PM Shubham Bansal wrote: > Hello everyone, >

Elastic scaling with HA (zookeeper)

2023-05-02 Thread Shubham Bansal
Hello everyone, Elastic scaling, as of now, has to be started in application mode. I am having some trouble in making it work with HA. I have a jobmanager pod and a single taskmanager pod, I use a post install helm hook to start the jobmanager along with the single job (As it's required in applica