FrankYang0529 commented on PR #17444: URL: https://github.com/apache/kafka/pull/17444#issuecomment-2410623953
> 1. An alternative would be use to MurmurHash for instance. What do you think? Agree, `hashCode()` is not good enough for it. We should use something like md5. Also, the hash function should generate same result for same data with different order, because different order doesn't impact target assignment result. > 2. I was hoping that we could actually get rid of the `Map<String, TopicMetadata>` data structure and have a hash which replaces all of it. Do you mean that we just want to have a hash value in `ConsumerGroupPartitionMetadataValue`, not a list of `TopicMetadata`? > Regarding option 1) in https://issues.apache.org/jira/browse/KAFKA-17747, have you discarded it? Since you mention controller in option 1, I'm not quite sure whether we want to store epoch in `TopicImage` or `TopicMetadata`. For a new epoch in `TopicImage`, we may need to store rack information to it as well, or the epoch can't represent all changes (e.g. rack change). However, the `TopicImage` is not only used in group coordinator, it's also used in KRaft. It's may not be a good idea to couple it. For a new epoch in `TopicMetadata`, we still need to calculate subscription metadata when `MetadataImage` is updated. To save storage resources, we don't want to store duplicated information. Eventually, the way may not be more efficient than option 2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
