Re: Failure to reset consumer offsets for specific topics

2017-10-25 Thread Elyahou Ittah
timestamp: 1508978010544 offset: 254410759 > > timestamp: 1508978011084 offset: 254410763 > > timestamp: 1508978012080 offset: 254410770 > > timestamp: 1508978012789 offset: 254410796 > > timestamp: 1508978013981 offset: 254410808 > > timestamp: 1508978014526 offset: 254410

Re: Failure to reset consumer offsets for specific topics

2017-10-25 Thread Elyahou Ittah
Which driver is used yo produce these messages ? On Oct 26, 2017 07:11, "Dan Markhasin" wrote: > No, that flag doesn't affect which offsets are returned, only executes the > action (and resets the consumer to latest offset when used, regardless of > datetime value I provide). > > On 25 October 2

Re: Incorrect consumer offsets after broker restart 0.11.0.0

2017-10-12 Thread Elyahou Ittah
Yes, this is the Jira ticket about this issue: https://issues.apache.org/jira/browse/KAFKA-5600 On Wed, Oct 11, 2017 at 5:47 PM, Vincent Dautremont < vincent.dautrem...@olamobile.com.invalid> wrote: > I would also like to know the related Jira ticket if any, to check that > what I experience the

Re: Incorrect consumer offsets after broker restart 0.11.0.0

2017-10-10 Thread Elyahou Ittah
It is a known bug, fixed in 0.11.0.1 On Oct 10, 2017 15:20, "Phil Luckhurst" wrote: > We have a Kafka broker we use for testing that we have recently updated > from 0.9.0.1 to 0.11.0.0 and our java consumer is built using the 0.11.0.0 > client. The consumers manually commit offsets and are consu

Re: Consumer reconsuming all kafka messages

2017-08-22 Thread Elyahou Ittah
It use the default one, which is uncomitted earliest On Wed, Aug 23, 2017 at 7:34 AM, Dan Markhasin wrote: > Is your storm consumer set to auto.offset.reset="earliest"? > > On 22 August 2017 at 10:05, Elyahou Ittah wrote: > > > I checked the __consumer_offsets top

Re: Consumer reconsuming all kafka messages

2017-08-22 Thread Elyahou Ittah
consumer group to rewind partition like this ? On Mon, Aug 21, 2017 at 11:10 AM, Elyahou Ittah wrote: > attached log file > > On Mon, Aug 21, 2017 at 11:06 AM, Elyahou Ittah > wrote: > >> I am consuming from kafka using KafkaSpout of Storm and also in ruby >> using ru

Re: Consumer reconsuming all kafka messages

2017-08-21 Thread Elyahou Ittah
attached log file On Mon, Aug 21, 2017 at 11:06 AM, Elyahou Ittah wrote: > I am consuming from kafka using KafkaSpout of Storm and also in ruby using > ruby-kafka gem (both use new consumer API). > > I noticed that after a rolling restart of the kafka cluster. The > kafkaSpout

Consumer reconsuming all kafka messages

2017-08-21 Thread Elyahou Ittah
I am consuming from kafka using KafkaSpout of Storm and also in ruby using ruby-kafka gem (both use new consumer API). I noticed that after a rolling restart of the kafka cluster. The kafkaSpout reconsumed all kafka messages ignoring the committed offsets... What can cause this behavior ? Attach