Hi Dan,
sorry for the mixup. I think the idleness definition [1] is orthogonal to
the used source interface. The new source interface just makes it more
obvious to the user that he can override the watermark strategy.
I'd still recommend having a look at the new Kafka source though. One
interesti
Are there any docs that talk about the new idleness support? I want to
understand it better.
https://ci.apache.org/projects/flink/flink-docs-release-1.13/docs/connectors/datastream/kafka/
On Thu, Jul 29, 2021 at 6:15 PM Dan Hill wrote:
> Thanks, JING and Arvid!
>
> Interesting. That's good to
Thanks, JING and Arvid!
Interesting. That's good to know. I've been planning for incompatible
schema changes. I'll look into new source too.
On Thu, Jul 29, 2021 at 4:56 AM Arvid Heise wrote:
> I'm assuming you have an incompatible schema change. If you don't, there
> are several easy ways.
I'm assuming you have an incompatible schema change. If you don't, there
are several easy ways.
Your plan actually looks like the best option. Of course, this requires
that you eventually union the inputs. If you can do that without a custom
mapper and with one read schema only, you may even use 1
Hi Dan,
Do you plan to continue to read a new Kafka topic after finished read
current Kafka topic?
If yes, Your plan could works.
BTW, if the schema of data in the new Kafka topic and the current topic are
same with each other, however their topic name are different with each
other, maybe you coul
Hi!
*Scenario*
I want to eventually do a breaking change to a Kafka source (major version
change) which requires a new Kafka topic.
*Question*
What utilities exist to help with this in Flink? What best practices exist?
My plan is roughly the following:
1. Change my Flink job to support both kaf
