Hi Nilesh,
For point 1, try by increasing the 'topology.message.timeout.secs' to 10
to 15 mins or more then slowly decrease it which suits your topology.
For me that worked for the same case.
For point 2, we have used database to made track what we have processed,
so don't process the same tuple again.
regards
Shams
On Thursday 06 November 2014 12:16 PM, Nilesh Chhapru wrote:
Hi All,
We are using storm Kafka integration where a Spout reads from a Kafka
topic.
Following is the version of storm, Kafka and zookeeper we are using.
*Strom : apache-storm-0.9.2-incubating*
*Kafka : kafka_2.8.0-0.8.1.1*
*Zookeeper : zookeeper-3.4.6*
I am facing following issues at spout.
1)The messages gets failed even if the average time taken is less than
max.topology.timeout value, also we aren’t getting any exceptions at
any of the bolt.
2)A topology is finally emitting to the Kafka producer i.e. some other
topic, but the messages are getting duplicated due to replay issues.
3)The consumer group is isn’t working properly for storm Kafka
integration.
a.When we give same group id to the Kafka consumer of different
topology but still both are reading same messages.
b.If we have 2 different consumer with different consumer group id in
different topology it works fine if both topologies are deployed at
the same time, but doesn’t if we deploy one of them after some of the
message are already loaded in the topic and read by the first topology.
Kindly help me with above points as it is hampering the overall scope
of the project and also time lines.
Do call or email in-case you need any other information.
*Nilesh Chhapru,*
(: +91 9619030491
------------------------------------------------------------------------
---------------------------------------------------------------------------------------Disclaimer----------------------------------------------------------------------------------------------
****Opinions expressed in this e-mail are those of the author and do
not necessarily represent those of Ugam. Ugam does not accept any
responsibility or liability for it. This e-mail message may contain
proprietary, confidential or legally privileged information for the
sole use of the person or entity to whom this message was originally
addressed. Any review, re-transmission, dissemination or other use of
or taking of any action in reliance upon this information by persons
or entities other than the intended recipient is prohibited. If you
have received this e-mail in error, please delete it and all
attachments from any servers, hard drives or any other media.
Warning: Sufficient measures have been taken to scan any presence of
viruses however the recipient should check this email and any
attachments for the presence of viruses. Ugam accepts no liability for
any damage caused by any virus transmitted by this email. ****
--
