That error comes from you calling create message stream twice or the container you are running in is causing this to be called twice https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/consumer/ ZookeeperConsumerConnector.scala#L133from , don't do that and you won't get the error.
On Mon, Dec 16, 2013 at 2:25 PM, Seshadri, Balaji <balaji.sesha...@dish.com>wrote: > Any update on this guys ?. > > -----Original Message----- > From: Seshadri, Balaji > Sent: Saturday, December 14, 2013 4:22 PM > To: users@kafka.apache.org > Subject: RE: Unable to start consumers in Tomcat > > We are doing one scala consumer and one java consumer who listen on same > topic with different group id.The issue is if I start both second one fails > with this error.If we run just one its working fine. > > Attaching Scala code and Java code where we call createMessageStreams.It > seems like issue with API not recognizing that its different group of > consumers that are started because createMessageStream doesnot accept > group.id but should refer already passed group.id to make sure second on > is new stream and not related even we listen on same topic. > > Please let me know if it is API issue or way we handle API. > > > -----Original Message----- > From: Joe Stein [mailto:joe.st...@stealth.ly] > Sent: Friday, December 13, 2013 4:36 PM > To: users@kafka.apache.org > Subject: Re: Unable to start consumers in Tomcat > > you do this at the consumer connector level not at the message stream level > > so one > propsA.put("group.id", "groupA") be > val configA = new ConsumerConfig(propsA) val one = Consumer.create(configA) > > and another > propsB.put("group.id", "groupB") be > val configB = new ConsumerConfig(propsB) val two = Consumer.create(configB) > > so you then manage each Consumer connector created by each of those > .create in separate threads (or actors or processes or however you are > doing concurrency) and you only call the create message stream each once > but each once on each of them > > makes sense? > > /******************************************* > Joe Stein > Founder, Principal Consultant > Big Data Open Source Security LLC > http://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ********************************************/ > > > On Fri, Dec 13, 2013 at 6:29 PM, Seshadri, Balaji > <balaji.sesha...@dish.com>wrote: > > > Cant we create message stream for same topic but different consumer > > group ?. > > > > -----Original Message----- > > From: Joe Stein [mailto:joe.st...@stealth.ly] > > Sent: Friday, December 13, 2013 4:23 PM > > To: users@kafka.apache.org > > Subject: Re: Unable to start consumers in Tomcat > > > > You should check your code to verify that it is only called once per > > instantiated consumer connector. Here is where the exception is > > thrown > > https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/con > > sumer/ZookeeperConsumerConnector.scala#L133from > > > > /******************************************* > > Joe Stein > > Founder, Principal Consultant > > Big Data Open Source Security LLC > > http://www.stealth.ly > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > > ********************************************/ > > > > > > On Fri, Dec 13, 2013 at 6:17 PM, Seshadri, Balaji > > <balaji.sesha...@dish.com>wrote: > > > > > We needed HTTP interface to start our consumers using REST interface > > > for management that why we chose tomcat to run our consumers. > > > > > > We create streams only once.when we initially start consumer. > > > > > > -----Original Message----- > > > From: Joe Stein [mailto:joe.st...@stealth.ly] > > > Sent: Friday, December 13, 2013 4:12 PM > > > To: users@kafka.apache.org > > > Subject: Re: Unable to start consumers in Tomcat > > > > > > That error comes if you are trying to create a message/stream on a > > > consumer instantiated object more than once. > > > > > > Why are you using tomcat for the consumers? Is it to see the results > > > of messages? if so you need to isolate the Consumer in some way so > > > there is a singleton (assuming one partition or if more than one > > > partition some type of consumer pool) and the messages being > > > consumed write to a thread safe memory structure and then the UI can > > > read from that thread safe memory structure. > > > > > > /******************************************* > > > Joe Stein > > > Founder, Principal Consultant > > > Big Data Open Source Security LLC > > > http://www.stealth.ly > > > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > > > ********************************************/ > > > > > > > > > On Fri, Dec 13, 2013 at 5:42 PM, Seshadri, Balaji > > > <balaji.sesha...@dish.com>wrote: > > > > > > > 0.8 > > > > > > > > -----Original Message----- > > > > From: Neha Narkhede [mailto:neha.narkh...@gmail.com] > > > > Sent: Friday, December 13, 2013 3:33 PM > > > > To: users@kafka.apache.org > > > > Subject: Re: Unable to start consumers in Tomcat > > > > > > > > Which version of kafka are you using? > > > > > > > > > > > > On Fri, Dec 13, 2013 at 2:29 PM, Seshadri, Balaji > > > > <balaji.sesha...@dish.com>wrote: > > > > > > > > > Any idea on this error guys ?. > > > > > > > > > > -----Original Message----- > > > > > From: Seshadri, Balaji [mailto:balaji.sesha...@dish.com] > > > > > Sent: Friday, December 13, 2013 9:32 AM > > > > > To: 'users@kafka.apache.org' > > > > > Subject: Unable to start consumers in Tomcat > > > > > > > > > > Hi , > > > > > > > > > > Can you guys lets us know why are we getting this error when try > > > > > to spawn a consumer ?. > > > > > > > > > > ZookeeperConsumerConnector can create message streams at most > > > > > once > > > > > > > > > > Balaji > > > > > > > > > > > > > > >
