Hi, We have a use case where we are consuming from more than 100s of Kafka Topics. Each topic has different number of partitions.
As per the documentation, to parallelize a Kafka Topic, we need to use setParallelism() == number of Kafka Partitions for a topic. But if we are consuming multiple topics in Flink by providing pattern eg. *my_topic_** and for each topic if there is different configuration for partitions, then how should we connect all these together so that we can map Kafka Partition to Flink Parallelization correctly and programmatically (so that we don't have to hard code all the topic names and parallelism -- considering we can access kafka topic <-> number of partitions mapping in Flink) ? Thanks,
