Re: New KafkaSource API: Change in default behavior regarding starting offset

2022-06-20 Thread Shengkai Fang
15/2022 17:50 > To Martijn Visser > Cc Jing Ge , > user > Subject Re: New KafkaSource API : Change in default behavior regarding > starting offset > Hello Martijn, > > Thanks for the link to the release note, especially : > "When resuming f

Re: New KafkaSource API: Change in default behavior regarding starting offset

2022-06-18 Thread liangzai
请问这个邮件咋退订? Replied Message | From | bastien dine | | Date | 06/15/2022 17:50 | | To | Martijn Visser | | Cc | Jing Ge, user | | Subject | Re: New KafkaSource API : Change in default behavior regarding starting offset | Hello Martijn, Thanks for the link to the release note

Re: New KafkaSource API : Change in default behavior regarding starting offset

2022-06-15 Thread bastien dine
Hello Martijn, Thanks for the link to the release note, especially : "When resuming from the savepoint, please use setStartingOffsets(OffsetsInitializer.committedOffsets()) in the new KafkaSourceBuilder to transfer the offsets to the new source." So earliest is the new default We use for sure .co

Re: New KafkaSource API : Change in default behavior regarding starting offset

2022-06-15 Thread Martijn Visser
Hi Bastien, When the FlinkKafkaConsumer was deprecated in 1.14.0, the release notes included the instruction how to migrate from FlinkKafkaConsumer to KafkaConsumer [1]. Looking at the Kafka documentation [2], there is a section on how to upgrade to the latest connector version that I think is out

Re: New KafkaSource API : Change in default behavior regarding starting offset

2022-06-15 Thread bastien dine
Hello jing, This was the previous method in old Kafka consumer API, it has been removed in 1.15, so source code is not in master anymore, Yes I know for the new Offset initializer, committed offset + earliest as fallback can be used to have the same behavior as before I just wanted to know whether

Re: New KafkaSource API : Change in default behavior regarding starting offset

2022-06-14 Thread Jing Ge
Hi Bastien, Thanks for asking. I didn't find any call of setStartFromGroupOffsets() within Flink in the master branch. Could you please point out the code that committed offset is used as default? W.r.t. the new KafkaSource, if OffsetsInitializer.committedOffsets() is used, an exception will be t

New KafkaSource API : Change in default behavior regarding starting offset

2022-06-14 Thread bastien dine
Hello everyone, Does someone know why the starting offset behaviour has changed in the new Kafka Source ? This is now from earliest (code in KafkaSourceBuilder), doc says : "If offsets initializer is not specified, OffsetsInitializer.earliest() will be used by default." from : https://nightlies.a