rtitions (for Kafka sources for example) and requires some coordination
> works.
>
> Best,
> Zhanghao Chen
> --
> *发件人:* Talat Uyarer via dev
> *发送时间:* 2023年7月23日 15:28
> *收件人:* dev
> *主题:* Scaling Flink Jobs without Restarting Job
>
> HI
nging.
Rescaling stateless job will still involve redistribution of Kafka partitions
(for Kafka sources for example) and requires some coordination works.
Best,
Zhanghao Chen
发件人: Talat Uyarer via dev
发送时间: 2023年7月23日 15:28
收件人: dev
主题: Scaling Flink Jobs wi
HI,
We are using Flink with Adaptive Scheduler(Reactive Mode) on Kubernetes
with Standalone deployment Application mode for our streaming
infrastructure. Our autoscaler is scaling up or down our jobs. However,
each scale action causes a job restart.
Our customers complain about fluctuating traffi
@Greg: Can you describe at what points the JobManager struggled heavily? I
would guess that it is at some point during deployment, that deployment
takes longer than you expected?
On Wed, Oct 21, 2015 at 10:14 AM, Maximilian Michels wrote:
> Hi Greg,
>
> It would be very interesting to do a profi
Hi Greg,
It would be very interesting to do a profiling of the job master to
see what it mostly spends time on. Did you run your experiments with
0.9.X or the 0.10-SNAPSHOT? Would be interesting to know if there is a
regression.
Best,
Max
On Wed, Oct 21, 2015 at 10:08 AM, Till Rohrmann wrote:
>
Hi Greg,
there is no official guide for running Flink on large clusters. As far as I
know, the cluster we used for the matrix factorization was the largest
cluster we've run a serious job on. Thus, it would be highly interesting to
understand what made the JobManager to slow down. At some point, t
Is there guidance for configuring Flink on large clusters? I have recently
been working to benchmark some algorithms on and test AWS. I had no issues
running on a 16 node cluster but when moving to 64 nodes the JobManager
struggled mightily. It did not look to be parallelizing its workload. I was
i
