Hello, I am trying to test the latency between message production and message consumption using Java Kafka-Client*(2.7.2)* library. The configuration of cluster is 3 KafkaBrokers*(2.7.2, Scala 2.13)*, 3 Zookeeper*(3.5.9)* Here is a pattern what I have observed Reference: ConsumerReadTimeStamp: Timestamp when record received in Kafka Consumer ProducerTimeStamp: Timestamp added before producer.send record RecordTimeStamp: CreateTimeStamp inside the record obtained at consumer
[image: kafka1.png] *For 100 Messages* *ConsumerReadTimeStamp-ProducerTimeStamp(ms)* *ConsumerReadTimeStamp-RecordTimeStamp(ms)* *Average* *252.56* *238.85* *Max* *2723* *2016* *Min* *125* *125* On the consumer side it takes too much time for initial few messages but later on it is quite consistent. I have executed the above same test for large number of messages : 100,1000,10000,etc. and the pattern seems to be same Here are the configurations, mostly using default properties. Topic: partitions=16 min.insync.replica=2 replication.factor=3 Consumer: security.protocol=PLAINTEXT enable.auto.commit=true Producer: security.protocol=PLAINTEXT compression.type=gzip acks=all Is there any reason why there is huge latency at the beginning when a consumer is created please? Also please suggest some way to optimise configurations to have some better consistent results ? Thank you in advance for your feedback.
