Joe Stein created KAFKA-1067:
Summary: the default partitioner should be randomizing messages
and a new partition for the meta refresh requirements created
Key: KAFKA-1067
URL: https://issues.apache.org/jira/browse/KA
Jun, can we hold this extra change over for 0.8.1 and just go with
reverting where we were before for the default with a new partition for
meta refresh and support both?
I am not sure I entirely understand why someone would need the extra
functionality you are talking about which sounds cool thoug
Joe,
Not sure I fully understand your propose. Do you want to put the random
partitioning selection logic (for messages without a key) in the
partitioner without changing the partitioner api? That's difficult. The
issue is that in the current partitioner api, we don't know which
partitions are ava
What I was proposing was two fold
1) revert the DefaultPartitioner class
then
2) create a new partitioner that folks could use (like at LinkedIn you
would use this partitioner instead) in ProducerConfig
private class RandomRefreshTimPartitioner[T](props: VerifiableProperties =
null) extends Par
However, currently, if key is null, the partitioner is not even called. Do
you want to change DefaultEventHandler too?
This also doesn't allow the partitioner to select a random and available
partition, which in my opinion is more important than making partitions
perfectly evenly balanced.
Thanks
hmmm, yeah, on I don't want todo that ... if we don't have to.
What if the DefaultPartitioner code looked like this instead =8^)
private class DefaultPartitioner[T](props: VerifiableProperties = null)
extends Partitioner[T] {
def partition(key: T, numPartitions: Int): Int = {
if (key == nu
We have the following code in DefaultEventHandler:
val partition =
if(key == null) {
// If the key is null, we don't really need a partitioner
// So we look up in the send partition cache for the topic to
decide the target partition
val id = sendPartitionPerTopicC
