Just an update, as I was reading about Kafka Streams, this functionality is by default supported with Kafka Streams Library. Following links are really helpful
http://docs.confluent.io/3.0.0/streams/developer-guide.html#partition-grouper https://github.com/apache/kafka/blob/0.10.0/streams/src/main/java/org/apache/kafka/streams/processor/DefaultPartitionGrouper.java (Kafka Stream is supported with 0.10.0) Thanks, Snehal On 5 July 2016 at 11:47, Snehal Nagmote <nagmote.sne...@gmail.com> wrote: > Hello Yardena , > > You may want to take a look at manual assignment for partitions section > mentioned here , > > http://www.confluent.io/blog/tutorial-getting-started-with-the-new-apache-kafka-0.9-consumer-client > . > > However I have not tried using this for multiple topics , but looking at > api , it should be doable. > > You have to use same partitioning method which was used by producer to > determine the correct partition for consumer process for multiple topics. > > Note that , you would lose the ordering guarantee with this approach since > Kafka guarantees ordering within partition for a single topic , > > Thanks, > Snehal > > > On 4 July 2016 at 07:50, Yardena Meymann <yarde...@feature.fm> wrote: > >> Hi, >> >> We have several topics, same number of partitions for each, same key used >> for all topics. >> We also have several processes consuming the topics (one consumer group). >> What we wish would happen is that messages with the same key would end up >> consumed by the same process, regardless of the topic. >> Can it be achieved with Kafka? What is needed for that? >> >> Thanks in advance, >> Yardena >> > >
