Re: [ANNOUNCE] Apache Flink 1.18.0 released

2023-10-29 Thread Tamir Sagi
Congratulations! thanks to the release managers and everyone who has contributed! Best Tamir From: Jark Wu Sent: Friday, October 27, 2023 7:39 AM To: d...@flink.apache.org Cc: Qingsheng Ren ; User ; user...@flink.apache.org Subject: Re: [ANNOUNCE] Apache Flin

Re:Delayed Window Trigger

2023-10-29 Thread Xuyang
Are you using Flink SQL? If using Flink SQL, the window is triggered when and only when the special data (with the expected timestamp after watermark) enters. It is not possible to trigger the window without changing the window-start and window-end column. -- Best! Xuyang At

Re: Bloom Filter for Rocksdb

2023-10-29 Thread David Anderson
I believe bloom filters are off by default because they add overhead and aren't always helpful. I.e., in workloads that are write heavy and have few reads, bloom filters aren't worth the overhead. David On Fri, Oct 20, 2023 at 11:31 AM Mate Czagany wrote: > Hi, > > There have been no reports ab

Re: Bloom Filter for Rocksdb

2023-10-29 Thread xiangyu feng
Hi Kean, I would like to share with you our analysis of the pros and cons about enabling Bloomfilter in production. Pros: By enabling BloomFilter, RocksDB.get() can filter out data files that not contains this key for sure and hence reduce some random disk reads. This performance improvement is d

Re: Enhancing File Processing and Kafka Integration with Flink Jobs

2023-10-29 Thread arjun s
Hi team, I appreciate the information provided. I'm inquiring whether there exists a method to automatically relocate processed files from a directory once a Flink job has completed processing them. I'm particularly keen on understanding how this particular use case is currently managed in product