Nilesh, Can you post the code you are using to setup the Kafka spout instances? Also your storm configuration would help as well.
-Taylor On Oct 31, 2014, at 4:04 AM, Nilesh Chhapru <[email protected]> wrote: > Hi All, > > Any update on below email, really stuck with this big time now L > > Regards, > Nilesh Chhapru. > > From: Nilesh Chhapru [mailto:[email protected]] > Sent: 28 October 2014 11:46 AM > To: [email protected]; [email protected]; [email protected] > Subject: Issues with storm kafka integration > > Hi Nathan, > > We are using storm Kafka integration where a Spout reads from a Kafka topic. > > Following is the version of storm, Kafka and zookeeper we are using. > Strom : apache-storm-0.9.2-incubating > Kafka : kafka_2.8.0-0.8.1.1 > Zookeeper : zookeeper-3.4.6 > > I am facing following issues at spout. > 1) The messages gets failed even if the average time taken is less than > max.topology.timeout value, also we aren’t getting any exceptions at any of > the bolt. > 2) A topology is finally emitting to the Kafka producer i.e. some other > topic, but the messages are getting duplicated due to replay issues. > 3) The consumer group is isn’t working properly for storm Kafka > integration. > a. When we give same group id to the Kafka consumer of different > topology but still both are reading same messages. > b. If we have 2 different consumer with different consumer group id in > different topology it works fine if both topologies are deployed at the same > time, but doesn’t if we deploy one of them after some of the message are > already loaded in the topic and read by the first topology. > > Kindly help me with above points as it is hampering the overall scope of the > project and also time lines. > > Do call or email in-case you need any other information. > > > Nilesh Chhapru, > (: +91 9619030491 > > > ---------------------------------------------------------------------------------------Disclaimer---------------------------------------------------------------------------------------------- > > ****Opinions expressed in this e-mail are those of the author and do not > necessarily represent those of Ugam. Ugam does not accept any responsibility > or liability for it. This e-mail message may contain proprietary, > confidential or legally privileged information for the sole use of the person > or entity to whom this message was originally addressed. Any review, > re-transmission, dissemination or other use of or taking of any action in > reliance upon this information by persons or entities other than the intended > recipient is prohibited. If you have received this e-mail in error, please > delete it and all attachments from any servers, hard drives or any other > media. > > Warning: Sufficient measures have been taken to scan any presence of viruses > however the recipient should check this email and any attachments for the > presence of viruses. Ugam accepts no liability for any damage caused by any > virus transmitted by this email. **** > > ---------------------------------------------------------------------------------------Disclaimer---------------------------------------------------------------------------------------------- > > ****Opinions expressed in this e-mail are those of the author and do not > necessarily represent those of Ugam. Ugam does not accept any responsibility > or liability for it. This e-mail message may contain proprietary, > confidential or legally privileged information for the sole use of the person > or entity to whom this message was originally addressed. Any review, > re-transmission, dissemination or other use of or taking of any action in > reliance upon this information by persons or entities other than the intended > recipient is prohibited. If you have received this e-mail in error, please > delete it and all attachments from any servers, hard drives or any other > media. > > Warning: Sufficient measures have been taken to scan any presence of viruses > however the recipient should check this email and any attachments for the > presence of viruses. Ugam accepts no liability for any damage caused by any > virus transmitted by this email. ****
signature.asc
Description: Message signed with OpenPGP using GPGMail
