The `KafkaTopicPartitionAssigner.assign(partition, numParallelSubtasks)` method
returns the index of the target subtask for a given Kafka partition.
The implementation in that method ensures that the same subtask index will
always be returned for the same partition.
Each consumer subtask will lo
The code of kafka partition assign is like follows:
public static int assign(KafkaTopicPartition partition, int
numParallelSubtasks) {
int startIndex = ((partition.getTopic().hashCode() * 31) &
0x7FFF) %
numParallelSubtasks;
// here, the assumption is that the
Hi!
You can set the parallelism of the Flink Kafka Consumer independent of the
number of partitions.
If there are more consumer subtasks than the number of Kafka partitions to read
(i.e. when the parallelism of the consumer is set higher than the number of
partitions), some subtasks will simply
It seems the same partition data will be consume multi times?
--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/
