Re: Preparing keyed state before snapshot

Hi Lorenzo, I think the most convenient way is to modify the code of the state backend, adding a k-v cache as you want. Otherwise IIUC, there's no public interface to get keyContext. But well, you may try something hacky. You may use the passed-in `Context` instance in processElement, and leverag

[ANNOUNCE] Apache Kyuubi 1.8.1 is available

Hi all, The Apache Kyuubi community is pleased to announce that Apache Kyuubi 1.8.1 has been released! Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses. Kyuubi provides a pure SQL gateway through Thrift JDBC/ODBC and RESTful inte

Entering a busy loop when adding a new sink to the graph + checkpoint enabled

Hello Guys, Can someone please assist us regarding the following issue ? We have noticed that when we add a *new kafka sink* operator to the graph, *and start from the last save point*, the operator is 100% busy for several minutes and *even 1/2-1 hour* !!! The problematic code seems to be the f

Re: Support for ConfigMap for Runtime Arguments in Flink Kubernetes Operator

Hi Arjun, Yes, direct support for external configuration files within Flink ConfigMaps is somewhat restricted. The current method involves simply copying two local files from the operator. Please check : FlinkConfMountDecorator#getLocalLogConfFiles()

Re: Preparing keyed state before snapshot

Thanks Zakelly, I'd need to do something similar, with a map containing my non-serializable "state", similar to the kvCache in FastTop1Fucntion. But I am not sure I understand how I can set the keyed state for a specific key, in snapshotState(). FastTop1Function seems to rely on keyContext set vi

Help in designing the Flink usecase

Classification: External Hi, I have a use case involving calculating the lifetime order count of a customer in real-time. To reduce the memory footprint, I plan to run a batch job on stored data every morning (let's say at 5 am) to calculate the total order count up to that moment. Additionally,

Community Over Code Asia 2024 Travel Assistance Applications now open!

Hello to all users, contributors and Committers! The Travel Assistance Committee (TAC) are pleased to announce that travel assistance applications for Community over Code Asia 2024 are now open! We will be supporting Community over Code Asia, Hangzhou, China July 26th - 28th, 2024. TAC exists to