Hello Jun, Thanks for your answer. I agree with you, I think problem is Storm 0.9 Spout.
Thanks, Gurkan On Thu, Jan 16, 2014 at 5:49 PM, Jun Rao <jun...@gmail.com> wrote: > If the Kafka consumption part is working, you would have to look into Storm > to see why the messages are dropped. > > Thanks, > > Jun > > > On Wed, Jan 15, 2014 at 11:32 PM, Gürkan Oluç <gurkano...@gmail.com> > wrote: > > > Hello, > > > > It looks like it consumes from Kafka. here is the request log : > > https://gist.github.com/gurkanoluc/39d2762ade4e2148a20c > > > > > > > > > > On Thu, Jan 16, 2014 at 7:29 AM, Jun Rao <jun...@gmail.com> wrote: > > > > > It seems that it's using SimpleConsumer. Could you check Kafka broker's > > > request log and see if the consumer is making the right fetch request? > > > > > > Thanks, > > > > > > Jun > > > > > > > > > On Wed, Jan 15, 2014 at 10:58 AM, Gürkan Oluç <gurkano...@gmail.com> > > > wrote: > > > > > > > Hello, > > > > > > > > We are using Kafka 0.8 with Storm. To consume from Kafka, we are > using > > > > https://github.com/wurstmeister/storm-kafka-0.8-plus this spout. We > > are > > > > creating spout with this configuration : > > > > https://gist.github.com/gurkanoluc/0cab49b8ddcfe15df8e9 > > > > > > > > Our problem is, consumers stops after working some time. From storm > > > logs, I > > > > am seeing that consumer offset is increasing. But, Storm is not > > emitting > > > > any thing to other bolts. > > > > > > > > Also, we had almost same issue with another classical java > application. > > > It > > > > stops after some time, but when I change the group id of consumer > group > > > or > > > > restart process it's starting to consume again. Changing group id > > > sometimes > > > > work for storm spout. My consumer configuration for this classic app > is > > > > https://gist.github.com/gurkanoluc/da46e7df263cfaffca16 > > > > > > > > I have looked to > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Myconsumerseemstohavestopped%2Cwhy%3Fthis > > > > topic from topic but, It doesn't help me. Is there anything to debug > > > > this issue? > > > > > > > > Instead of consuming with storm, If I consume with kafka's own > scripts > > to > > > > consume. It consumes successfully. > > > > > > > > Thanks for your help. > > > > Best, > > > > > > > > -- > > > > Gürkan OLUÇ > > > > http://www.gurkanoluc.com > > > > > > > > > > > > > > > -- > > Gürkan OLUÇ > > http://www.gurkanoluc.com > > > -- Gürkan OLUÇ http://www.gurkanoluc.com
