Other thing is if you are using SimpleConsumer, it is up to your app to do the offsetManagement. The ZK based offsets or Kafka based offsets will work if you are using the HighLevel Consumer.
Thanks, Mayuresh On Fri, Mar 27, 2015 at 9:17 AM, Mayuresh Gharat <gharatmayures...@gmail.com > wrote: > Hi Madhukar, > > I am going through your code now. Let me see what I can find. > > Where were you storing your offsets before? > Was it always Zookeeper or was it Kafka? > If it was Zookeeper, the correct way to migrate from zookeeper to kafka > based offsets is this : > > 1) Config Change : > - offsets.storage = kafka > - dual.commit.enabled = true > 2) Rolling Bounce > 3) Config Change : > - dual.commit.enabled=false > 4) Rolling Bounce. > > For more info on Offset Management, you can also refer these slides from > Kafka Meetup: > http://www.slideshare.net/jjkoshy/offset-management-in-kafka > > > Apart from that for using Kafka based offsets, to do a fetchOffsetRequest > or commit offset request you don't need a consumer. You need to know the > groupId. You need to connect to kafka, issue a consumerMetaData Request. > This will fetch you the OffsetManager for that groupId. You can then issue > the fetch or commit request to that OffsetManager. > > BTW, we are coming up with an offsetClient soon. > > Thanks, > > Mayuresh > > On Fri, Mar 27, 2015 at 1:53 AM, Madhukar Bharti <bhartimadhu...@gmail.com > > wrote: > >> Hi Mayuresh, >> >> Please check this >> <https://github.com/madhukarbharti/kafka-8.2.1-test/blob/master/src/com/bharti/kafka/offset/OffsetRequester.java> >> program. >> Am I doing any mistake? >> >> Thanks >> >> >> On Thu, Mar 26, 2015 at 6:27 PM, Madhukar Bharti < >> bhartimadhu...@gmail.com> wrote: >> >>> Hi Mayuresh, >>> >>> I have tried to fetch the offset using OffsetFetchRequest as given in >>> this wiki >>> >>> >>> https://cwiki.apache.org/confluence/display/KAFKA/Committing+and+fetching+consumer+offsets+in+Kafka >>> >>> But It only works if we set "dual.commit.enabled" to "true" and >>> "offsets.storage" to "kafka". Otherwise it returns -1. >>> >>> Do I need to change anything? >>> >>> >>> Thanks in advance! >>> >> >> > > > -- > -Regards, > Mayuresh R. Gharat > (862) 250-7125 > -- -Regards, Mayuresh R. Gharat (862) 250-7125
