No, I played a bit to check differences. I tried to send messages in both ways:
But using just ONE keys (Subject0) for ALL the messages the situation was: >>>> bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list >>>> localhost:9092 --topic LCacc --time -1 >>>> LCacc:8:60699 >>>> LCacc:2:60737 >>>> LCacc:5:60716 >>>> LCacc:4:60722 >>>> LCacc:7:60701 >>>> LCacc:1:60744 >>>> LCacc:9:60695 >>>> LCacc:3:60731 >>>> LCacc:6:60710 >>>> LCacc:0:61967 So quietly fair distributed. while I expected to find all the messages in one partition. Then, I tried to use many keys to verify if something was changed and the distribution was essentially the same (as I expected). Andrea > On 23 Oct 2017, at 17:54, eazama...@gmail.com wrote: > > Please correct me if I've misunderstood something, but your keys being > Subject0, Subject1, Subject2, etc, mean you don't actually have identical > keys, right? If that's the case then you wouldn't expect to see all of the > messages in a single partition. > >> On Oct 23, 2017, at 8:43 AM, Andrea Giordano <andrea.giordano....@gmail.com> >> wrote: >> >> The output of: >> ./kafka_2.11-0.11.0.0/bin/kafka-topics.sh --zookeeper localhost:2181 --list >> >> LCacc >> LCgyr >> LTacc >> LTgyr >> RCacc >> RCgyr >> RTacc >> RTgyr >> __consumer_offsets >> >> output sample for a topic (all equals) >> ./kafka_2.11-0.11.0.0/bin/kafka-topics.sh --describe --zookeeper >> localhost:2181 --topic LCacc >> >> Topic:LCacc PartitionCount:10 ReplicationFactor:1 Configs: >> Topic: LCacc Partition: 0 Leader: 0 Replicas: 0 Isr: 0 >> Topic: LCacc Partition: 1 Leader: 0 Replicas: 0 Isr: 0 >> Topic: LCacc Partition: 2 Leader: 0 Replicas: 0 Isr: 0 >> Topic: LCacc Partition: 3 Leader: 0 Replicas: 0 Isr: 0 >> Topic: LCacc Partition: 4 Leader: 0 Replicas: 0 Isr: 0 >> Topic: LCacc Partition: 5 Leader: 0 Replicas: 0 Isr: 0 >> Topic: LCacc Partition: 6 Leader: 0 Replicas: 0 Isr: 0 >> Topic: LCacc Partition: 7 Leader: 0 Replicas: 0 Isr: 0 >> Topic: LCacc Partition: 8 Leader: 0 Replicas: 0 Isr: 0 >> Topic: LCacc Partition: 9 Leader: 0 Replicas: 0 Isr: 0 >> >> the kafka producer is implemented in node-red, so the code about it is in >> javascript. >> look at the link for it: https://pastebin.com/Rqd2Q3kx >> >> I use Flink java to get kafka messages and implementing a deserialiser >> effectively I see my key string in key kafka message field (so I’m “quite” >> sure the javascript code is correctly implemented). >> >> Finally my keys are something like: >> >> Subject0 >> Subject1 >> Subject2 >> … and so on. >> >> >> Thank you for the help, >> Andrea >> >> >> >> >>> On 23 Oct 2017, at 17:25, Manikumar <manikumar.re...@gmail.com> wrote: >>> >>> Can you post the sample code? >>> >>> On Mon, Oct 23, 2017 at 8:53 PM, Andrea Giordano < >>> andrea.giordano....@gmail.com> wrote: >>> >>>> Hi, >>>> I set a Kafka broker with some topics and where each topic is divided into >>>> 10 partitions. >>>> As I understood on Kafka doc, if I send a keyed message to kafka the key >>>> is used to distribute the message in a particular partition (using an hash >>>> value of the key): in this way messages with same key go in the same >>>> partition. >>>> >>>> I’m trying to run the broker sending messages with the same key and I >>>> noted that executing the following command: >>>> >>>> bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list >>>> localhost:9092 --topic LCacc --time -1 >>>> >>>> >>>> I obtained the output: >>>> >>>> Honestly I expected to see just a partition full (due to the only one key >>>> used). Instead it seems kafka distributed message in a quite uniform way. >>>> Is this normal? >>>> I tried also to send messages with different keys (for example sending >>>> much more messages with a key respect to another) but nothing changed. >>>> >>>> LCacc:8:60699 >>>> LCacc:2:60737 >>>> LCacc:5:60716 >>>> LCacc:4:60722 >>>> LCacc:7:60701 >>>> LCacc:1:60744 >>>> LCacc:9:60695 >>>> LCacc:3:60731 >>>> LCacc:6:60710 >>>> LCacc:0:61967 >>>> >>>> >>
