Re: No Heartbeat request on commit

2016-04-28 Thread vinay sharma
I sent it as attachment. I did not zip it and may have been rejected by mail server. Will send again shortly. On Apr 28, 2016 5:11 PM, "Jason Gustafson" wrote: > Hey Vinay, > > Did you forget to attach the simple class? > > -Jason > > On Thu, Apr 28, 2016 at 1:14 PM, vinay sharma > wrote: > > >

Re: No Heartbeat request on commit

2016-04-28 Thread Jason Gustafson
Hey Vinay, Did you forget to attach the simple class? -Jason On Thu, Apr 28, 2016 at 1:14 PM, vinay sharma wrote: > I was also wondering that if commitSync acts as heartbeat then why do we > still trigger heartbeat request on commit? why not just reset its time on > successful commitSync? or a

Re: No Heartbeat request on commit

2016-04-28 Thread vinay sharma
I was also wondering that if commitSync acts as heartbeat then why do we still trigger heartbeat request on commit? why not just reset its time on successful commitSync? or am i wrong and we do this already? Regards, Vinay On Thu, Apr 28, 2016 at 3:38 PM, vinay sharma wrote: > Hi Jason, > > Att

Re: No Heartbeat request on commit

2016-04-28 Thread vinay sharma
Hi Jason, Attached is a simple class with a main method. I used this for reproducing issue and generate logs that i attached earlier. This class has code snippets of poller relevant to the issue. Regards, Vinay Sharma On Thu, Apr 28, 2016 at 3:30 PM, Jason Gustafson wrote: > Hey Vinay, > > Tha

Re: No Heartbeat request on commit

2016-04-28 Thread Jason Gustafson
Hey Vinay, Thanks, that's really helpful. It does seem like there might be a problem with the heartbeat trigger logic. I'll see if I can reproduce what you're seeing locally. Might be helpful if you share a snippet of your poll loop. Thanks, Jason On Thu, Apr 28, 2016 at 11:55 AM, vinay sharma

Re: No Heartbeat request on commit

2016-04-28 Thread vinay sharma
Hi Jason, i reverted back to KAFKA-3149. Producer still had issues related to schema but my consumer worked. Now consumer worked as expected. Although i did not encountered an error and generation was not marked dead by coordinator but i still see that successful heartbeat response are not logged

Re: No Heartbeat request on commit

2016-04-28 Thread Jason Gustafson
Ah, yeah. That's probably caused by the new topic metadata version, which isn't supported on 0.9 brokers. To test on trunk, you'd have to upgrade the brokers as well. Either that or you can rewind to before KAFKA-3306 (which was just committed the day before yesterday)? -Jason On Thu, Apr 28, 201

Re: No Heartbeat request on commit

2016-04-28 Thread vinay sharma
Hi Jason, I build kafka-client and tried using it but my producers and consumers started throwing below exception. Is 0.10 not going to be compatible with brokers on version 0.9.0.1? or do i need to make some config changes to producers / consumers to make them compatible with brokers on old versi

Re: No Heartbeat request on commit

2016-04-27 Thread Jason Gustafson
Hey Vinay, Any chance you can run the same test against trunk? I'm guessing this might be caused by a bug in the 0.9 consumer which basically causes some requests to fail when a bunch of them are sent to the broker at the same time. -Jason On Wed, Apr 27, 2016 at 1:02 PM, vinay sharma wrote: >

Re: No Heartbeat request on commit

2016-04-27 Thread Jason Gustafson
Hi Vinay, Answers below: 1) Is it correct to say that each commitSync will trigger a HeartBeatTask? > If there is no hear beat sent in past since specified heartbeat interval > then i should see a successful heartbeat response or failure message in > logs near to commitSync success log? Not qu

Re: No Heartbeat request on commit

2016-04-27 Thread vinay sharma
Hey, I am working on a simplified test case to check if there is any issue in my code. Just to make sure that any of my assumptions are not wrong, it will be great if you can please help me in finding answers to following queries:- 1) Is it correct to say that each commitSync will trigger a Hear

Re: No Heartbeat request on commit

2016-04-26 Thread Jason Gustafson
Hey Vinay, Are you saying that heartbeats are not sent while a metadata refresh is in progress? Do you have any logs which show us the apparent problem? Thanks, Jason On Tue, Apr 26, 2016 at 8:18 AM, vinay sharma wrote: > Hi Ismael, > > Treating commitSync as heartbeat will definitely resolve

Re: No Heartbeat request on commit

2016-04-26 Thread vinay sharma
Hi Ismael, Treating commitSync as heartbeat will definitely resolve the issue i am facing but the reason behind my issue does not seem to be what mentioned in defect (i.e frequent commitSync requests). I am sending CommitSync periodically only to keep my session alive when my consumer is still pr

Re: No Heartbeat request on commit

2016-04-25 Thread Ismael Juma
Hi Vinay, This was fixed via https://issues.apache.org/jira/browse/KAFKA-3470 (will be part of 0.10.0.0). Ismael On Mon, Apr 25, 2016 at 1:52 PM, vinay sharma wrote: > Hello, > > I am using client API 0.9.0.1 and facing an issue. As per my logs it seems > that on each commitSync(Offsets) a h

No Heartbeat request on commit

2016-04-25 Thread vinay sharma
Hello, I am using client API 0.9.0.1 and facing an issue. As per my logs it seems that on each commitSync(Offsets) a heartbeat request is sent but after a metada refresh request till next poll(), commits do not send any hearbeat request. KafkaConsumers i create sometimes get session time out due