Re: Flink pipeline throughput

2024-04-01 Thread Asimansu Bera
Hello Kartik, For your case, if events ingested/Second is 300/60=5 and payload size is 2kb , per second, ingestion size 5*2k=10kb. Network buffer size is 32kb by default. You can also decrease the value to 16k. https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/deployment/config/#tas

Re: Flink pipeline throughput

2024-04-01 Thread Kartik Kushwaha
Thank you. I will check and get back on both the sugesstions made by Asimansu and Xuyang. I am using Flink 1.17.0 Regards, Kartik On Mon, Apr 1, 2024, 5:13 AM Asimansu Bera wrote: > Hello Karthik, > > You may check the execution-buffer-timeout-interval parameter. This value > is an important o

Re: Flink pipeline throughput

2024-03-31 Thread Asimansu Bera
Hello Karthik, You may check the execution-buffer-timeout-interval parameter. This value is an important one for your case. I had a similar issue experienced in the past. https://nightlies.apache.org/flink/flink-docs-release-1.18/docs/deployment/config/#execution-buffer-timeout-interval For your