Re: Is there anyway to control the size of my ListState in my KeyedProcessFunction

2025-05-29 Thread Zakelly Lan
Hi Sachin, I assume you are using the rocksdb state backend. The TTL for ListState is applied for each list entry, if you are using `ListState.add`. However if you do ListState.update, the entire list is rewrite so the ttl is updated. Could you share your use case and the ttl config? Another sugge

Unsubscribe

2025-05-29 Thread HyungHo Kim
Unsubscribe This communication contains Aeris Confidential Information. If it has been sent to you in error, please reply to advise the sender of the error and then immediately delete this message.

Help with Azure Workload Identity

2025-05-29 Thread Ralph Blaise
Hi all, I am having the hardest time trying to use an Azure DataLake Gen2 Storage Account using Azure Workload Identity with abfss scheme with Flink 1.20 or 2.0. I keep on getting that either org.apache.hadoop.fs.azurebfs.oauth2.WorkloadIdentityTokenProvider not found or java.lang.IllegalArgum

Re: Is there anyway to control the size of my ListState in my KeyedProcessFunction

2025-05-29 Thread Sachin Mittal
Hi, I think ttl would be applied for the entire list, I would like the ListState to restrict the entries by size and automatically purge older added entries as new ones get added. Something similar to a bounded list. Thanks Sachin On Thu, May 29, 2025 at 6:51 PM Sigalit Eliazov wrote: > hi, >

Is there anyway to control the size of my ListState in my KeyedProcessFunction

2025-05-29 Thread Sachin Mittal
Hi, I am adding some hashes of my elements in my list date to check for dedups. Now to not have an infinte growing list, I would like to limit the size of hashes in that list to say a number or just add some TTL config which would expire the entries in the list beyond certain time. Is this someth