The key is used by the client to decide which partition to send the message to. By the time the client is creating the produce request, it should be known which partition each message is being sent to. I believe Neha described the behavior of the Java client which sends messages with a null key to any partition.
The key is described in past tense because of the use case for persisting keys with messages. The key is persisted through the broker so that a consumer knows what key was used to partition the message on the producer side. I don't believe that you can have the broker decide which partition a message goes to. -- Colin B. On 05/21/2013 11:48 AM, Dave Peterson wrote: > I'm looking at the document entitled "A Guide to the Kafka Protocol" > located here: > > https://cwiki.apache.org/KAFKA/a-guide-to-the-kafka-protocol.html > > It shows a produce request as containing a number of message sets, which are > grouped first by topic and second by partition (a 32-bit integer). > However, each > message in a message set contains a Key field, which is described as follows: > > The key is an optional message key that was used for partition assignment. > The key can be null. > > I notice the use of "was" (past tense) above. That seems to suggest that the > Key field was once used to specify a partition (at the granularity of each > individual message), but the plan for the future is to instead use the 32-bit > partition value preceding each message set. Is this correct? If so, when I > am > creating a produce request for 0.8, what should I use for the 32-bit partition > value, and how does this relate to the Key field of each individual message? > Ideally, I would like to just send a produce request and let the broker choose > the partition. How do I accomplish this in 0.8, and are there plans to change > this after 0.8? > > Thanks, > Dave > > On Tue, May 21, 2013 at 10:47 AM, Neha Narkhede <neha.narkh...@gmail.com> > wrote: >> No. In 0.8, if you don't specify a key for a message, it is sent to any of >> the available partitions. In other words, the partition id is selected on >> the partition and the server doesn't get -1 as the partition id. >> >> Thanks, >> Neha >> >> >> On Tue, May 21, 2013 at 9:54 AM, Dave Peterson <dspeter...@tagged.com>wrote: >> >>> In the version 0.8 wire format for a produce request, does a value of -1 >>> still indicate "use a random partition" as it did for 0.7? >>> >>> Thanks, >>> Dave >>>
