you do this at the consumer connector level not at the message stream level
so one
propsA.put("group.id", "groupA") be
val configA = new ConsumerConfig(propsA)
val one = Consumer.create(configA)
and another
propsB.put("group.id", "groupB") be
val configB = new ConsumerConfig(propsB)
val two = Consumer.create(configB)
so you then manage each Consumer connector created by each of those
.create in separate threads (or actors or processes or however you are
doing concurrency) and you only call the create message stream each once
but each once on each of them
makes sense?
/*******************************************
Joe Stein
Founder, Principal Consultant
Big Data Open Source Security LLC
http://www.stealth.ly
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
********************************************/
On Fri, Dec 13, 2013 at 6:29 PM, Seshadri, Balaji
<balaji.sesha...@dish.com>wrote:
> Cant we create message stream for same topic but different consumer group
> ?.
>
> -----Original Message-----
> From: Joe Stein [mailto:joe.st...@stealth.ly]
> Sent: Friday, December 13, 2013 4:23 PM
> To: users@kafka.apache.org
> Subject: Re: Unable to start consumers in Tomcat
>
> You should check your code to verify that it is only called once per
> instantiated consumer connector. Here is where the exception is thrown
> https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/consumer/ZookeeperConsumerConnector.scala#L133from
>
> /*******************************************
> Joe Stein
> Founder, Principal Consultant
> Big Data Open Source Security LLC
> http://www.stealth.ly
> Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> ********************************************/
>
>
> On Fri, Dec 13, 2013 at 6:17 PM, Seshadri, Balaji
> <balaji.sesha...@dish.com>wrote:
>
> > We needed HTTP interface to start our consumers using REST interface
> > for management that why we chose tomcat to run our consumers.
> >
> > We create streams only once.when we initially start consumer.
> >
> > -----Original Message-----
> > From: Joe Stein [mailto:joe.st...@stealth.ly]
> > Sent: Friday, December 13, 2013 4:12 PM
> > To: users@kafka.apache.org
> > Subject: Re: Unable to start consumers in Tomcat
> >
> > That error comes if you are trying to create a message/stream on a
> > consumer instantiated object more than once.
> >
> > Why are you using tomcat for the consumers? Is it to see the results
> > of messages? if so you need to isolate the Consumer in some way so
> > there is a singleton (assuming one partition or if more than one
> > partition some type of consumer pool) and the messages being consumed
> > write to a thread safe memory structure and then the UI can read from
> > that thread safe memory structure.
> >
> > /*******************************************
> > Joe Stein
> > Founder, Principal Consultant
> > Big Data Open Source Security LLC
> > http://www.stealth.ly
> > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
> > ********************************************/
> >
> >
> > On Fri, Dec 13, 2013 at 5:42 PM, Seshadri, Balaji
> > <balaji.sesha...@dish.com>wrote:
> >
> > > 0.8
> > >
> > > -----Original Message-----
> > > From: Neha Narkhede [mailto:neha.narkh...@gmail.com]
> > > Sent: Friday, December 13, 2013 3:33 PM
> > > To: users@kafka.apache.org
> > > Subject: Re: Unable to start consumers in Tomcat
> > >
> > > Which version of kafka are you using?
> > >
> > >
> > > On Fri, Dec 13, 2013 at 2:29 PM, Seshadri, Balaji
> > > <balaji.sesha...@dish.com>wrote:
> > >
> > > > Any idea on this error guys ?.
> > > >
> > > > -----Original Message-----
> > > > From: Seshadri, Balaji [mailto:balaji.sesha...@dish.com]
> > > > Sent: Friday, December 13, 2013 9:32 AM
> > > > To: 'users@kafka.apache.org'
> > > > Subject: Unable to start consumers in Tomcat
> > > >
> > > > Hi ,
> > > >
> > > > Can you guys lets us know why are we getting this error when try
> > > > to spawn a consumer ?.
> > > >
> > > > ZookeeperConsumerConnector can create message streams at most once
> > > >
> > > > Balaji
> > > >
> > >
> >
>
