Re: Flink parallelism with Kafka source

2025-01-02 Thread David Anderson
Flink handles its parallelism independently from the number of partitions in the topic(s) being read. The parallelism comes from whatever is set in the cluster configuration, without any concern for the source's native parallelism. If there are fewer kafka partitions than the flink parallelism, the

Flink parallelism with Kafka source

2024-12-20 Thread Guillermo Ortiz Fernández
I'm looking for how Flink defines parallelism for a Kafka source ( https://nightlies.apache.org/flink/flink-docs-master/docs/connectors/table/kafka/). How is it determined by default? Is it based on the number of partitions in the topic? I have some topics with hundreds of partitions, and such a hi