Re: Samza partition hashing relative to other clients

2022-12-15 Thread Malcolm McFarland
Hi Stuart, Thanks for the tip. I've come across this configuration as well. It's not so much an issue that I want to change Kafka's partitioner, rather that I want to ensure that Samza is using Kafka's default partitioner (although your point about explicitly defining the same partitioner in every

Re: Samza partition hashing relative to other clients

2022-12-14 Thread Stuart Perks
Hi Malcolm, You should be able to override the following producer config for partitioner.class: https://kafka.apache.org/24/documentation.html#producerconfigs This can be done as follows via Samza config systems.system-name.prod

Samza partition hashing relative to other clients

2022-12-14 Thread Malcolm McFarland
Hey folks, I'm working on a system where several different Kafka clients (including Samza) are producing into the same Kafka topic. It's necessary for each of these clients to calculate the same partition hash for the same key input to ensure consistent message ordering (there are some asynchronou