Using Kafka Streams this is a little tricky. The API itself has no built-in mechanism to do this. You would need to monitor the lag of the application, and if the lag is zero (assuming you don't write new data into the topic in parallel), terminate the application.
-Matthias On 9/14/18 4:19 AM, Henning Røigaard-Petersen wrote: > Spin up a consumer, subscribe to EOF events, assign all partitions from the > beginning, and keep polling until all partitions has reached EOF. > Though, if you have concurrent writers, new messages may be appended after > you observe EOF on a partition, so you are never guaranteed to have read all > messages at the time you choose to close the consumer. > > /Henning Røigaard-Petersen > > -----Original Message----- > From: David Espinosa <espi...@gmail.com> > Sent: 14. september 2018 09:46 > To: users@kafka.apache.org > Subject: Best way for reading all messages and close > > Hi all, > > Although the usage of Kafka is stream oriented, for a concrete use case I > need to read all the messages existing in a topic and once all them has been > read then closing the consumer. > > What's the best way or framework for doing this? > > Thanks in advance, > David, >
signature.asc
Description: OpenPGP digital signature
