This ticket describes how the default partitioner works (and links to the source code) https://issues.apache.org/jira/browse/KAFKA-3333
On Thu, Mar 31, 2016 at 9:34 AM, Marcelo Oikawa <marcelo.oik...@webradar.com > wrote: > Hi, everyone. > > If you don't specify the partition, and do have a key, then the default > > behaviour is to use a hash on the key to determine the partition. This to > > make sure the messages with the same key and up on the same partition. > This > > helps to ensure ordering relative to the key/partition. Also when using > > compaction instead of delete as cleanup policy the newest messages with > the > > same key are kept. This is also used for the internal __offset topic. > > > > I didnt know exactly if the messages with same key are placed in the same > partition. > Thats help me a lot. Thanks. > > Do you know if the hash funcion is MD5? Just curiours. > > > > I'll have > > > to look at the documentation but I'm not entirely sure if the consumers > > > have access to this key. > > > > Actually, they have access to the key[1]. > > [1] http://grokbase.com/t/kafka/users/135944gyke/key-used-by-producer > > > > The producer does. You can override the default > > > partitioner class and write one that uses your understands and > interprets > > > your definition of the key to place data in a specific partition. By > > > default, I believe data is distributed using a round robin partitioner. > > > > > > > > > > > > On Thu, Mar 31, 2016 at 2:58 AM, Marcelo Oikawa < > > > marcelo.oik...@webradar.com > > > > wrote: > > > > > > > Hi, list. > > > > > > > > We're working on a project that uses Kafka and we notice that for > every > > > > message we have a key (or null). I searched for more info about the > key > > > > itself and the documentation says that it is only used to decide the > > > > partition where the message is placed. > > > > > > > > Is there a problem if we use keys with the application semantics > > > > (metadata)? For instance, we can use the key "origin:foo;target:boo" > > and > > > > the consumers may use the key info to make decisions. But, a lot of > > > > messages may use the same key and it may produce unbalanced > partitions, > > > is > > > > that right? > > > > > > > > Does anyone know more about the key and your role inside kafka? > > > > > > > > []s > > > > > > > > > > > > > > > > -- > > > -- > > > Sharninder > > > > > >
