Hi Team, I'm using Kafka partitioning to maintain field-based ordering across partitions, but I'm experiencing data skewness among the partitions. I have 96 partitions, and I'm sending data with 500 distinct keys that are used for partitioning. While monitoring the Kafka cluster, I noticed that a few partitions are underutilized while others are overutilized.
This seems to be a hashing problem. Can anyone suggest a better hashing technique or partitioning strategy to balance the load more effectively? Thanks in advance for your help.
