Best Practices? Fault Isolation for Processing Large Number of Same-Shaped Input Kafka Topics in a Big Flink Job

Hi everyone, I'm currently prototyping on a project where we need to process a large number of Kafka input topics (say, a couple of hundred), all of which share the same DataType/Schema. Our objective is to run the same Flink SQL on all of the input topics, but I am concerned about doing this in

Re: Can we use custom serialization/deserialization for kafka sources and sinks through the table api?

Hi Gabriel, You could consider overriding the value.serializer and value.deserializer (and similar for key) in the consumer and producer configu

Re: Resume running a statefull in a different k8s cluster

one thing you could consider is a mutator that detects when a failover is happening, and then updates the CR to point to the right snapshot to restore from. https://nightlies.apache.org/flink/flink-kubernetes-operator-docs-release-1.12/docs/operations/plugins/#custom-flink-resource-mutators On Th