Flink 1.18 support for flink stateful functions

2024-04-12 Thread Deshpande, Omkar via user
Hello, Is there a plan to add support for flink 1.18 in flink stateful function? Also, generally the stateful functions seem to be slow and lag behind the flink release cycle. Is the stateful function project going to be actively maintained? Thanks, Omkar

Pyflink w Nessie and Iceberg in S3 Jars

2024-04-12 Thread Robert Prat
Hi there, For several days I have been trying to find the right configuration for my pipeline which roughly consists in the following schema RabbitMQ->PyFlink->Nessie/Iceberg/S3. For what I am going to explain I have tried both locally and through the official Flink docker images. I have trie

Re: Understanding event time wrt watermarking strategy in flink

2024-04-12 Thread Sachin Mittal
Hi Yunfeng, I have a question around the tolerance for out of order bound watermarking, What I understand that when consuming from source with out of order bound set as B, lets say it gets a record with timestamp T. After that it will drop all the subsequent records which arrive with the timestamp

Re: Understanding event time wrt watermarking strategy in flink

2024-04-12 Thread Yunfeng Zhou
Hi Sachin, 1. When your Flink job performs an operation like map or flatmap, the output records would be automatically assigned with the same timestamp as the input record. You don't need to manually assign the timestamp in each step. So the windowing result in your example should be as you have e