The other option is jmx bean that exposes the lag. Also Kafka provides at least once guarantees so even if your consumer lags occasionally, you will eventually receive all mesaages. You need to provision enough consumers so that they don't fall behind.
Thanks, Neha On May 23, 2013 5:30 AM, "arathi maddula" <arathimadd...@gmail.com> wrote: > Hi Neha, > Thanks for the quick reply. Could you tell me if there is some way of > determining the offset for a consumer from a high level Java consumer class > apart from ConsumerOffsetChecker tool? This tool can be run only from the > command line. Is it possible to use this in a Java class? > > I write streaming data from a website into Kafka topic and then read the > same data using a servlet and serve it to a Java client. The problem is > that the servlet sometimes does not consume data from the topic as quickly > as it is produced. In order to give warning to the user saying data is > lost, I will need some way of determining the consumer group offset from > the servlet. > > The servlet is a high level Kafka consumer and it is sufficient for our > needs, so I don't want to use a simple consumer. > > Any pointers will be of great help. > Thanks > Arathi > On Wed, May 22, 2013 at 7:24 PM, Neha Narkhede <neha.narkh...@gmail.com > >wrote: > > > You can run the ConsumerOffsetChecker tool that ships with Kafka. > > > > Thanks, > > Neha > > > > > > On Wed, May 22, 2013 at 2:02 PM, arathi maddula <arathimadd...@gmail.com > > >wrote: > > > > > Hi, > > > > > > Could you tell me how to find the offset in a high level Java > consumer ? > > > > > > Thanks > > > Arathi > > > > > >
