Thanks. This makes sense. Jamie
-----Original Message----- From: Neha Narkhede [mailto:neha.narkh...@gmail.com] Sent: Monday, January 28, 2013 5:27 PM To: users@kafka.apache.org Subject: Re: How do you keep track of offset in a partition Jamie, You need to use the getOffsetsBefore() API to get the earliest/latest offset available on the broker for a particular partition. Thanks, Neha On Mon, Jan 28, 2013 at 5:05 PM, Jamie Wang <jamie.w...@actuate.com> wrote: > Hi, > > We are using 0.72 version of Kafka on Windows. I am wondering what is the > right way to fetch data and keep track of offset in a partition. For > example, I am currently assuming the first message the producer sent to the > broker is at offset 0. So far it seems working. Is this correct assumption? > > Let' say 2 days later, the first 100 messages on the broker is discarded > because it passed retention.hours set in the config file. Now what is the > offset I should use to retrieve the first message in the partition? And > let's also say the offset I had for the 80th message is now not valid. > What is the right way to get the correct offset to fetch in the consumer? > > What is the purpose of the api for getting a list of valid offsets for all > segments in a partition? > > Thanks in advance for your help. > > Jamie >
