Re: Backpressure handling in FileSource APIs - Flink 1.16

Hello Shammon, Can you please point out the classes where like for "FileSource" slow down logic is placed? Just wanted to understand it more better and try it at my end by running various perf. runs, also apply changes in my application if any. Rgds, Kamal On Wed, May 24, 2023 at 11:41 AM Kamal

Web UI don't show up In Flink on Yarn (Flink 1.17)

Hi all,   I find a strange thing with flink 1.17 deployed on yarn (CDH 6.x), flink web ui can not show up from yarn web link "ApplicationMaster",even typed jobmanager ip directly in browser . when i run wordcount application in flink 1.17 examples, and click yarn web "ApplicationMaster" link

Re: Why I can't run more than 19 tasks?

Hi Hemi, There may be two reasons that I can think of 1. The number of connections exceeds the MySQL limit, you can check the options in my.cnf for your mysql server and increase the max connections. 2. Connection timeout for mysql client, you can try to add 'autoReconnect=true' to the connection

Why I can't run more than 19 tasks?

hey everybody, I have a problem with my apache flink, I am synchronizing from MySQL to Elasticsearch but it seems that I can't run more than 19 tasks. it gave me this error: -- Caused by: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.util.FlinkRuntimeException: java.sql.SQLTra

Re: Kafka Quotas & Consumer Group Client ID (Flink 1.15)

Hi Hatem, The reason for setting different client ids is to due to Kafka client metrics conflicts and the issue is documented here: https://nightlies.apache.org/flink/flink-docs-stable/docs/connectors/datastream/kafka/#kafka-consumer-metrics. I think that the warning log is benign if you are using

Re: Kafka Quotas & Consumer Group Client ID (Flink 1.15)

Hello Martijn, Yes, checkpointing is enabled and the offsets are committed without a problem. I think I might have figured out the answer to my second question based on my understanding of this code

First Flink Embedded Stateful Functions taking long to get invoked

Hi team, We are exploring flink stateful function for one of our use case. As part of feasibility test, we are doing load testing to determine the time spent by flink app orchestrating the functions (E2E request latency - ti

Re: Kafka Quotas & Consumer Group Client ID (Flink 1.15)

Hi Hatem, Could it be that you don't have checkpointing enabled? Flink only commits its offset when a checkpoint has been completed successfully, as explained on https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/datastream/kafka/#consumer-offset-committing Best regards, Martij

Re: Reading KafkaSource state from a savepoint using the State Processor API

Hi, Charles, I am used to read the state in the debug mode. I always set the breakpoint at the return statemnet in `SavepointReader#read`. Then I could find the state I need in the field `SavepointMetadataV2 savepointMetadata`. Finally I could deserialize the state bytes with `KafkaPartitionSplitS