wow, that’s probable.
Excuse me but I’m a javascript noob. I understood I have to edit line 55:
var producer = new HighLevelProducer(client);
using the template:
HighLevelProducer(client, [options], [customPartitioner])
where partitionerType is the option.
but how to insert the option in it?
var producer = new HighLevelProducer(client, partitionerType:3);
sounds too strange to me…
thank you
Andrea
> On 23 Oct 2017, at 18:01, Manikumar <manikumar.re...@gmail.com
> <mailto:manikumar.re...@gmail.com>> wrote:
>
> looks like you need to use keyedPartitioner option(partitionerType =3).
> default is cyclic partitioning.
>
> https://www.npmjs.com/package/kafka-node#highlevelproducer
> <https://www.npmjs.com/package/kafka-node#highlevelproducer>
>
> On Mon, Oct 23, 2017 at 9:24 PM, <eazama...@gmail.com
> <mailto:eazama...@gmail.com>> wrote:
>
>> Please correct me if I've misunderstood something, but your keys being
>> Subject0, Subject1, Subject2, etc, mean you don't actually have identical
>> keys, right? If that's the case then you wouldn't expect to see all of the
>> messages in a single partition.
>>
>>> On Oct 23, 2017, at 8:43 AM, Andrea Giordano <
>> andrea.giordano....@gmail.com <mailto:andrea.giordano....@gmail.com>> wrote:
>>>
>>> The output of:
>>> ./kafka_2.11-0.11.0.0/bin/kafka-topics.sh --zookeeper localhost:2181
>> --list
>>>
>>> LCacc
>>> LCgyr
>>> LTacc
>>> LTgyr
>>> RCacc
>>> RCgyr
>>> RTacc
>>> RTgyr
>>> __consumer_offsets
>>>
>>> output sample for a topic (all equals)
>>> ./kafka_2.11-0.11.0.0/bin/kafka-topics.sh --describe --zookeeper
>> localhost:2181 --topic LCacc
>>>
>>> Topic:LCacc PartitionCount:10 ReplicationFactor:1 Configs:
>>> Topic: LCacc Partition: 0 Leader: 0 Replicas: 0 Isr: 0
>>> Topic: LCacc Partition: 1 Leader: 0 Replicas: 0 Isr: 0
>>> Topic: LCacc Partition: 2 Leader: 0 Replicas: 0 Isr: 0
>>> Topic: LCacc Partition: 3 Leader: 0 Replicas: 0 Isr: 0
>>> Topic: LCacc Partition: 4 Leader: 0 Replicas: 0 Isr: 0
>>> Topic: LCacc Partition: 5 Leader: 0 Replicas: 0 Isr: 0
>>> Topic: LCacc Partition: 6 Leader: 0 Replicas: 0 Isr: 0
>>> Topic: LCacc Partition: 7 Leader: 0 Replicas: 0 Isr: 0
>>> Topic: LCacc Partition: 8 Leader: 0 Replicas: 0 Isr: 0
>>> Topic: LCacc Partition: 9 Leader: 0 Replicas: 0 Isr: 0
>>>
>>> the kafka producer is implemented in node-red, so the code about it is
>> in javascript.
>>> look at the link for it: https://pastebin.com/Rqd2Q3kx
>>> <https://pastebin.com/Rqd2Q3kx>
>>>
>>> I use Flink java to get kafka messages and implementing a deserialiser
>> effectively I see my key string in key kafka message field (so I’m “quite”
>> sure the javascript code is correctly implemented).
>>>
>>> Finally my keys are something like:
>>>
>>> Subject0
>>> Subject1
>>> Subject2
>>> … and so on.
>>>
>>>
>>> Thank you for the help,
>>> Andrea
>>>
>>>
>>>
>>>
>>>> On 23 Oct 2017, at 17:25, Manikumar <manikumar.re...@gmail.com
>>>> <mailto:manikumar.re...@gmail.com>> wrote:
>>>>
>>>> Can you post the sample code?
>>>>
>>>> On Mon, Oct 23, 2017 at 8:53 PM, Andrea Giordano <
>>>> andrea.giordano....@gmail.com <mailto:andrea.giordano....@gmail.com>>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>> I set a Kafka broker with some topics and where each topic is divided
>> into
>>>>> 10 partitions.
>>>>> As I understood on Kafka doc, if I send a keyed message to kafka the
>> key
>>>>> is used to distribute the message in a particular partition (using an
>> hash
>>>>> value of the key): in this way messages with same key go in the same
>>>>> partition.
>>>>>
>>>>> I’m trying to run the broker sending messages with the same key and I
>>>>> noted that executing the following command:
>>>>>
>>>>> bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list
>>>>> localhost:9092 --topic LCacc --time -1
>>>>>
>>>>>
>>>>> I obtained the output:
>>>>>
>>>>> Honestly I expected to see just a partition full (due to the only one
>> key
>>>>> used). Instead it seems kafka distributed message in a quite uniform
>> way.
>>>>> Is this normal?
>>>>> I tried also to send messages with different keys (for example sending
>>>>> much more messages with a key respect to another) but nothing changed.
>>>>>
>>>>> LCacc:8:60699
>>>>> LCacc:2:60737
>>>>> LCacc:5:60716
>>>>> LCacc:4:60722
>>>>> LCacc:7:60701
>>>>> LCacc:1:60744
>>>>> LCacc:9:60695
>>>>> LCacc:3:60731
>>>>> LCacc:6:60710
>>>>> LCacc:0:61967
>>>>>
>>>>>
>>>
>>
