Hi ajay,
When you have 3 parallelisms you will have 3 independent clients. If you
want to keep prefetch count 3 you need to set setRequestedChannelMax as 1
and setParallelism 3. So All 3 clients can have one connection.
Talat
On Tue, May 7, 2024 at 5:52 AM ajay pandey wrote:
> Hi Flink Team,
>
Talat Uyarer created FLINK-33530:
Summary: Add entropy to Google Cloud Storage path for better
scalability
Key: FLINK-33530
URL: https://issues.apache.org/jira/browse/FLINK-33530
Project: Flink
rtitions (for Kafka sources for example) and requires some coordination
> works.
>
> Best,
> Zhanghao Chen
> --
> *发件人:* Talat Uyarer via dev
> *发送时间:* 2023年7月23日 15:28
> *收件人:* dev
> *主题:* Scaling Flink Jobs without Restarting Job
>
> HI
HI,
We are using Flink with Adaptive Scheduler(Reactive Mode) on Kubernetes
with Standalone deployment Application mode for our streaming
infrastructure. Our autoscaler is scaling up or down our jobs. However,
each scale action causes a job restart.
Our customers complain about fluctuating traffi
ers.
> > This way users can also integrate easily with the custom Flink metrics
> too.
> >
> > maxReplicas: We could add this easily to the taskManager resource specs
> >
> > Nice workflow picture, I would love to include this in the docs later.
> One
> >
we might need some integration with the k8s metrics
>> system.
>> In any case whether we need a FLIP or not depends on the complexity, if
>> it's simple then we can go without a FLIP.
>>
>> Cheers,
>> Gyula
>>
>> On Tue, May 24, 2022 at 12:26 PM T
on the complexity, if
> it's simple then we can go without a FLIP.
>
> Cheers,
> Gyula
>
> On Tue, May 24, 2022 at 12:26 PM Talat Uyarer <
> tuya...@paloaltonetworks.com> wrote:
>
>> Hi Gyula,
>>
>> This seems very promising for initial scaling. W
auto-scaler would do).
>>
>> Looking forward to your thoughts
>>
>> [1] https://github.com/tillrohrmann/flink/commits/autoscaling
>> <https://urldefense.com/v3/__https://github.com/tillrohrmann/flink/commits/autoscaling__;!!Mt_FR42WkD9csi9Y!ZNBiCduZFUmuQI7_9M48gQ
Hi,
I am working on auto scaling support for native deployments. Today Flink
provides Reactive mode however it only runs on standalone deployments. We
use Kubernetes native deployment. So I want to increase or decrease job
resources for our streamin jobs. Recent Flip-138 and Flip-160 are very
usefu
