Re: Flink consuming rate increases slowly

2020-05-13 Thread Dawid Wysakowicz
Hi Eyal, Could you explain your job a bit more? Did you increase the parallelism of your job? What does it do? Does it perform any time based operations? How do you measure the processing rate? Best, Dawid On 10/05/2020 21:18, Chen Qin wrote: > Hi Eyal, > > It’s unclear what warmup phase does i

Re: Flink consuming rate increases slowly

2020-05-10 Thread Chen Qin
Hi Eyal, It’s unclear what warmup phase does in your use cases. Usually we see Flink start consume at high rate and drop to a point downstream can handle. Thanks Chen > On May 10, 2020, at 12:25 AM, Eyal Pe'er wrote: > > Hi all, > Lately I've added more resources to my Flink cluster which req

Flink consuming rate increases slowly

2020-05-10 Thread Eyal Pe'er
Hi all, Lately I've added more resources to my Flink cluster which required a restart of all apps. >From the cluster side, the only change I made, is to add more task slots. On the cluster I have a streaming app that consumes from Kafka and sinks to files. I noticed that since the restart, the ap