Incompatible KafkaProducer version

2023-09-10 Thread Krzysztof Jankiewicz
Hi, I am currently working on a simple application that requires exactly-once end-to-end guarantee. I am reading data from Kafka and writing it back to Kafka. When I use `DeliveryGuarantee.AT_LEAST_ONCE` at the Kafka Sink level, everything works fine. Here's the relevant code: KafkaSink sink =

Re: Problem when testing table API in local

2023-09-10 Thread liu ron
Hi, Why don't you add the flink-table-planner module as dependency directly for test. Best, Ron Oscar Perez via user 于2023年9月8日周五 21:44写道: > quick update, > > after adding flink-clients it *worked *the first time I ran the test but > then the second time got the same error. Looks like a race

Re: [DISCUSS][FLINK-31788][FLINK-33015] Add back Support emitUpdateWithRetract for TableAggregateFunction

2023-09-10 Thread Feng Jin
Thanks Jane for following up on this issue! +1 for adding it back first. Supporting emitUpdateWithRetract for TableAggregateFunction is a good feature, we should support it unless there are better alternatives. Best, Feng On Thu, Sep 7, 2023 at 11:01 PM Lincoln Lee wrote: > Thanks to Jane fo

Re: using CheckpointedFunction on a keyed state

2023-09-10 Thread liu ron
Hi Krzysztof Can you give the original code in initializeState method and the corresponding exception stack trace? It looks a little interesting. Best, Ron Krzysztof Chmielewski 于2023年9月9日周六 07:12写道: > My apologies Mattthias, > you are right. The issue was that I was trying to access state

Re: Watermarks

2023-09-10 Thread liu ron
Hi, Sid For the second question, I think it is not needed. Best, Ron Feng Jin 于2023年9月9日周六 21:19写道: > hi Sid > > > 1. You can customize KafkaDeserializationSchema[1], in the `deserialize` > method, you can obtain the Kafka event time. > > 2. I don't think it's necessary to explicitly mention

Re: Order of Execution in KeyedBroadcastProcessFunction

2023-09-10 Thread David Anderson
In Flink, all user functions, including KeyedBroadcastProcessFunction, are (effectively) single threaded, so the processBroadcastElement method will run to completion before any further messages are processed in the processElement method. (I said "effectively" because in the case of processing time

Re: Async IO metrics for tps

2023-09-10 Thread patricia lee
Hi, I just would like to confirm if using richAsyncFunction not recommended to increment metrcs? Because the counter I have created does not correctly increment its value. But it works using the RichMapFunction as an example in the documentation. I am using RichAsyncFunction to send data to exter