If the Kafka consumption part is working, you would have to look into Storm to see why the messages are dropped.
Thanks, Jun On Wed, Jan 15, 2014 at 11:32 PM, Gürkan Oluç <gurkano...@gmail.com> wrote: > Hello, > > It looks like it consumes from Kafka. here is the request log : > https://gist.github.com/gurkanoluc/39d2762ade4e2148a20c > > > > > On Thu, Jan 16, 2014 at 7:29 AM, Jun Rao <jun...@gmail.com> wrote: > > > It seems that it's using SimpleConsumer. Could you check Kafka broker's > > request log and see if the consumer is making the right fetch request? > > > > Thanks, > > > > Jun > > > > > > On Wed, Jan 15, 2014 at 10:58 AM, Gürkan Oluç <gurkano...@gmail.com> > > wrote: > > > > > Hello, > > > > > > We are using Kafka 0.8 with Storm. To consume from Kafka, we are using > > > https://github.com/wurstmeister/storm-kafka-0.8-plus this spout. We > are > > > creating spout with this configuration : > > > https://gist.github.com/gurkanoluc/0cab49b8ddcfe15df8e9 > > > > > > Our problem is, consumers stops after working some time. From storm > > logs, I > > > am seeing that consumer offset is increasing. But, Storm is not > emitting > > > any thing to other bolts. > > > > > > Also, we had almost same issue with another classical java application. > > It > > > stops after some time, but when I change the group id of consumer group > > or > > > restart process it's starting to consume again. Changing group id > > sometimes > > > work for storm spout. My consumer configuration for this classic app is > > > https://gist.github.com/gurkanoluc/da46e7df263cfaffca16 > > > > > > I have looked to > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped%2Cwhy%3Fthis > > > topic from topic but, It doesn't help me. Is there anything to debug > > > this issue? > > > > > > Instead of consuming with storm, If I consume with kafka's own scripts > to > > > consume. It consumes successfully. > > > > > > Thanks for your help. > > > Best, > > > > > > -- > > > Gürkan OLUÇ > > > http://www.gurkanoluc.com > > > > > > > > > -- > Gürkan OLUÇ > http://www.gurkanoluc.com >
