Re: Queue implementation

2016-03-30 Thread Helleren, Erik
I don¹t follow. By having two consumer objects on C3, you can consume a portion of the messages from both T1 and T2. So, Group1(C1,C2,C3) is subscribed to topic Topic T1. Group2(C3,C4) is subscribed to topic T2 If you want C3 to consume all messages on T1 and T2, it would need to be in a unique

Re: Queue implementation

2016-03-29 Thread Vinod Kakad
Erik, But the two instances of consumer object each with a unique group-Id will deliver the message to both the consumer instances and I don't want that. Is there anyway to implement the use case I have mentioned or the use case itself does not fit kafka? On Tue, Mar 29, 2016 at 10:20 PM, Hel

Re: Queue implementation

2016-03-29 Thread Helleren, Erik
You could create two instances of the Consumer object, each with a unique group ID. On 3/29/16, 11:10 AM, "Gaurav Agarwal" wrote: >I am not aware of any. > >On Mar 29, 2016 6:14 PM, "Vinod Kakad" wrote: >> >> But can one consumer have 2 group-id's, cause in my question I have >> mentioned that

Re: Queue implementation

2016-03-29 Thread Gaurav Agarwal
I am not aware of any. On Mar 29, 2016 6:14 PM, "Vinod Kakad" wrote: > > But can one consumer have 2 group-id's, cause in my question I have > mentioned that c3 will be subscribed to both T1 and T2. > > So, can there be two groups like Group1(C1,C2,C3) for Topic T1 and > Group2(C3,C4) for Topic T

Re: Queue implementation

2016-03-29 Thread Vinod Kakad
But can one consumer have 2 group-id's, cause in my question I have mentioned that c3 will be subscribed to both T1 and T2. So, can there be two groups like Group1(C1,C2,C3) for Topic T1 and Group2(C3,C4) for Topic T2? is it possible? how can I achieve implementation like this? On Tue, Mar 29,

Re: Queue implementation

2016-03-29 Thread Gaurav Agarwal
Yes this is possible in Kafka. While make a connection with Kafka consumer . We can have same consumer group id for the same topic So while connecting topic T1 with 3 consumer c1,c2,c3 all will have the same group id so only one of the consumer will receive the message at time. Same we can do wi

Re: Queue implementation

2016-03-28 Thread Manikumar Reddy
Yes. your scenarios are easy to implement using Kafka. Pl go through Kafka documentation and examples for better understanding of Kafka concepts, use cases and design. https://kafka.apache.org/documentation.html https://github.com/apache/kafka/tree/trunk/examples On Tue, Mar 29, 2016 at 9:20 AM,

Re: Queue implementation

2016-03-28 Thread Vinod Kakad
Can we have multiple topics and multiple consumers but in a queue implementation. for e.g. There are 2 topics T1 and T2 and four consumers C1,C2,C3 and C4. can we have C1,C2,C3 subscribe to T1 and (C3,C4) subscribe to T2. and this should work like a queue. i.e. T2 should send message to only C3

Re: Queue implementation

2016-03-28 Thread Gaurav Agarwal
If you have consumer group id across multiple consumers then Kafka will work as queue only . On Mar 28, 2016 6:48 PM, "Sharninder" wrote: > What kind of queue are you looking for? Kafka works as a nice FIFO queue by > default anyway. > > > > On Mon, Mar 28, 2016 at 5:19 PM, Vinod Kakad wrote: >

Re: Queue implementation

2016-03-28 Thread Achanta Vamsi Subhash
If you want 1:1, start only one subscriber for a topic from the app. Then it will act as a queue. On Mon, Mar 28, 2016 at 6:48 PM, Sharninder wrote: > What kind of queue are you looking for? Kafka works as a nice FIFO queue by > default anyway. > > > > On Mon, Mar 28, 2016 at 5:19 PM, Vinod Kaka

Re: Queue implementation

2016-03-28 Thread Sharninder
What kind of queue are you looking for? Kafka works as a nice FIFO queue by default anyway. On Mon, Mar 28, 2016 at 5:19 PM, Vinod Kakad wrote: > Can anybody share any good example(code) for kafka as a queue > implementation? > > Thanks & Regards, > Vinod Kakad. > -- -- Sharninder