produce latency or producer -> consumer latency?
150ms is very high. Have a look at the
kafka.network:type=RequestMetrics,name=TotalTimeMs,request=Produce broker
metric which is the total message produce time. This will also account for
most of the end-to-end latency the broker is responsible for,
You can use Kafka very effectively from Python. There are a number of
Python clients to choose from, the most popular ones being
confluent-kafka-python, kafka-python and pykafka. The biggest thing missing
for non-Java developers is Kafka Streams, though there is a very promising
project underway to
confluent-kafka-python is built on librdkafka which can do kerberos
security.
https://github.com/confluentinc/confluent-kafka-python
some notes on how to configure it are here:
https://github.com/edenhill/librdkafka/wiki/Using-SASL-with-librdkafka
On Thu, Apr 13, 2017 at 6:36 PM, Yuxiang Mai
Two ideas:
you could use a new consumer group id and set the TopicConfig property
"auto.offset.reset" to "smallest". Consumers in the new group will read
from the beginning on all partitions.
Alternatively, as an example of how to set the offset explicitly, you can
modify the AdvancedConsumer exa
