RE: FlinkKafkaConsumer -> KafkaSource State Migration

2021-11-01 Thread Schwalbe Matthias
Thanks Fabian, That was the information I was missing. (Late reply ... same here, FlinkForward 😊 ) Thias -Original Message- From: Fabian Paul Sent: Donnerstag, 28. Oktober 2021 08:38 To: Schwalbe Matthias Cc: Mason Chen ; user Subject: Re: FlinkKafkaConsumer -> KafkaSou

Re: FlinkKafkaConsumer -> KafkaSource State Migration

2021-10-27 Thread Fabian Paul
Hi, Sorry for the late reply but most of use were involved in the Flink Forward conference. The upgrade strategies for the Kafka sink and source are pretty similar. Source and sink do not rely on state migration but leveraging Kafka as source of truth. When running with FlinkKafkaConsumer Maso

RE: FlinkKafkaConsumer -> KafkaSource State Migration

2021-10-27 Thread Schwalbe Matthias
I would also be interested on instructions/discussion on how to state-migrate from pre-unified sources/sinks to unified ones (Kafka) 😊 Thias From: Mason Chen Sent: Mittwoch, 27. Oktober 2021 01:52 To: user Subject: FlinkKafkaConsumer -> KafkaSource State Migration Hi all, I read th

FlinkKafkaConsumer -> KafkaSource State Migration

2021-10-26 Thread Mason Chen
Hi all, I read these instructions for migrating to the KafkaSource: https://nightlies.apache.org/flink/flink-docs-release-1.14/release-notes/flink-1.14/#deprecate-flinkkafkaconsumer . Do we need to employ any uid/allowNonRestoredState tricks if our Flink job is also stateful outside of the source