Hey Prashant, the Kafka Consumer parallelism is constrained by the number of partitions the topic(s) have. If you have configured the Kafka Consumer in Flink with a parallelism of 100, but your topic has only 20 partitions, 80 consumer instances in Flink will be idle.
On Mon, Apr 26, 2021 at 2:54 AM Prashant Deva <prash...@astradot.com> wrote: > Can a Kafka Consumer Source have more tasks run in parallel than the > number of partitions for the topic it is the source of? Or is the max > parallelism of the source constrained by max partitions of the topic? > > > >
