Hello,
I am using Kafka broker and Java client library v 0.11.0.0.
When I restart my Kafka consumer application which uses Java Kafka client
library to retrieve messages, I notice that for each partition, the message
associated with the last offset that was committed successfully gets
re-consumed
For my use case, I need to figure out the lag within the Java consumer
itself that is consuming some topic. Ideally, the consumer application
would monitor the lag every minute or so and take some action on its own if
consumer falls behind (i.e. spin up more threads to process records - my
use case
NM. 0.11 KafkaConsumer seems to have added "endOffsets" API!
On Thu, Oct 12, 2017 at 3:31 PM, Manan G wrote:
> For my use case, I need to figure out the lag within the Java consumer
> itself that is consuming some topic. Ideally, the consumer application
> would monitor the
Hello,
If KafkaConsumer is subscribed to more than one topic or even for same
topic, if the consumer is assigned more than one partition, what is the
behavior of KafkaConsumer.poll()?
In our use case, we would like to use, for example, "user id" as a key for
records for topics. Naturally, for som
Hello,
I am using 0.11.0.0 version of Kakfa broker and Java client library. My
consumer code tracks offsets for each assigned partition and at some time
interval manually commits offsets by specifying partition->offset map.
What I noticed is, after the rebalance, even if consumer loses some
parti
