Re: Offsets getting lost if no messages sent for a long time

2016-08-23 Thread Gerard Klijs
he dry period > (period without a new message is not well defined in advance). > > > > > > Regards, > > Rahul Misra > > > > > > -----Original Message----- > > From: Misra, Rahul [mailto:rahul.mi...@altisource.com] > > Sent: Sunday, August 21, 2016 12:4

Re: Offsets getting lost if no messages sent for a long time

2016-08-23 Thread Michael Freeman
t; > -Original Message- > From: Misra, Rahul [mailto:rahul.mi...@altisource.com] > Sent: Sunday, August 21, 2016 12:46 AM > To: Ian Wrigley; users@kafka.apache.org > Subject: RE: Offsets getting lost if no messages sent for a long time > > Hi Ian, > > Thanks for t

RE: Offsets getting lost if no messages sent for a long time

2016-08-22 Thread Misra, Rahul
Misra -Original Message- From: Misra, Rahul [mailto:rahul.mi...@altisource.com] Sent: Sunday, August 21, 2016 12:46 AM To: Ian Wrigley; users@kafka.apache.org Subject: RE: Offsets getting lost if no messages sent for a long time Hi Ian, Thanks for the quick response. Your answer clears

RE: Offsets getting lost if no messages sent for a long time

2016-08-20 Thread Misra, Rahul
message is not well defined in advance). Regards, Rahul Misra -Original Message- From: Ian Wrigley [mailto:i...@confluent.io] Sent: Sunday, August 21, 2016 12:01 AM To: users@kafka.apache.org Subject: Re: Offsets getting lost if no messages sent for a long time Since nothing was wri

Re: Offsets getting lost if no messages sent for a long time

2016-08-20 Thread Ian Wrigley
Since nothing was written to the __consumer_offsets topic for more than its configured retention period (offsets.retention.minutes, by default 1440 minutes, or one day), the offset info will be removed. Retention period is all about when the last offset was written, not the last time a Consumer

Offsets getting lost if no messages sent for a long time

2016-08-20 Thread Misra, Rahul
Hi, I have observed the following scenario (the consumer here has 'enable.auto.commit=false' and offsets are committed using commitSync() if any messages are received): 1. Start a consumer (with a specific group.Id) and send some messages to its subscribed topic. 2. The consumer con