Re: Write-time based synchronization of message consumption from multiple topics

2015-08-20 Thread David KOCH
Hello Petr, Thank you for your reply. I am not sure how your proposed solution could help my use case. Sending messages with the same key to the same partition will ensure that key : (message_type, value) that share a key will be consumed by the same consumer instance. However, my main concern wa

Re: Write-time based synchronization of message consumption from multiple topics

2015-08-20 Thread Petr Novak
Hi, to my knowledge this is generally solved by partitioner which logs messages with the same key to the same topic. It is not clear why you can't use this approach. With Regards, Petr On Thu, Aug 20, 2015 at 12:35 PM, David KOCH wrote: > Hello, > > Is it possible to consume messages from multi

Write-time based synchronization of message consumption from multiple topics

2015-08-20 Thread David KOCH
Hello, Is it possible to consume messages from multiple topics in a way that messages which were written at the same time in different topics are also consumed at around the same time? Let me explain this with our use case. Our application logs keyed messages of N different types into N topics: