Re: Completeablefuture in a flat map operator

2024-02-19 Thread Lasse Nedergaard
Hi In my case I have to query an external system. The system returns n numbers of row in a page and I have to call the system until no more data. I could use AsyncFunction if I buffer all records and output at the end but I want to “stream” and don’t have enough memory to hold all the data. AsyncFu

New Relic alerts for Flink job submission failure and recovery

2024-02-19 Thread elakiya udhayanan
Hi Team, I would like to know the possibilities of configuring the new relic alerts for a Flink job whenever the job is submitted, gets failed and recovers from the failure. In our case, we have configured the Flink environment as a Kubernetes pod running on an EKS cluster and the application code

Re: Completeablefuture in a flat map operator

2024-02-19 Thread Ken Krugler
Is there some reason why you can’t use an AsyncFunction? https://nightlies.apache.org/flink/flink-docs-master/api/java/org/apache/flink/streaming/api/functions/async/AsyncFunction.html Note that when dealing with event time and exactly once, an AsyncFunction provides required support for proper

Completeablefuture in a flat map operator

2024-02-19 Thread Lasse Nedergaard
Hi. I have a case where I would like to collect object from a completeablefuture future in a flat map function. I run into some problem where I get an exception regarding a buffer pool that don’t exists when I collect the objets after some times. I can see if I for testing don’t return from

Re: Impact of RocksDB backend on the Java heap

2024-02-19 Thread Alexis Sarda-Espinosa
Hi Zakelly, Yeah that makes sense to me, I was just curious about whether reading could be a bottleneck or not, but I imagine user-specific logic would be better than a generic cache from Flink that might habe a low hit rate. Thanks again, Alexis. On Mon, 19 Feb 2024, 07:29 Zakelly Lan, wrote:

Support for ConfigMap for Runtime Arguments in Flink Kubernetes Operator

2024-02-19 Thread arjun s
Hi team, I am currently in the process of deploying Flink on Kubernetes using the Flink Kubernetes Operator and have encountered a scenario where I need to pass runtime arguments to my Flink application from a properties file. Given the dynamic nature of Kubernetes environments and the need for fl