Re: Time difference between message fetch and message send is very high (~900-1200ms)

[Francis Dallaire]

What are the settings you have set on the producer ... is there any sync ack ? 
If so which type ?

In any case, it would be useful to know the parameters you set on the consumer 
and producer.

Thanks

Frank


Sent from Samsung Mobile



-------- Original message --------
From: "Kishore V. Kopalle" <kish...@greenmedsoft.com>
Date: 05-17-2013 2:11 AM (GMT-05:00)
To: users@kafka.apache.org,Francis Dallaire <francis.dalla...@ubisoft.com>
Subject: Re: Time difference between message fetch and message send is very 
high (~900-1200ms)


Hi Frank,

Yes, they are on the same machine. Zoo Keeper, Kafka server and the 
KafkaConsumerProducerDemo (with one producer and one consumer)  are running on 
the same computer.

Regards,
Kishore

On Fri, May 17, 2013 at 11:38 AM, Francis Dallaire 
<francis.dalla...@ubisoft.com<mailto:francis.dalla...@ubisoft.com>> wrote:
Quick question. Are the producer and consumer on the same machine ? If not, are 
you accounting for any difference in the time  setting between producer and 
consumer ?

Frank

Sent from Samsung Mobile



-------- Original message --------
From: Kishore Kopalle <kkopa...@gmail.com<mailto:kkopa...@gmail.com>>
Date: 05-17-2013 1:41 AM (GMT-05:00)
To: users@kafka.apache.org<mailto:users@kafka.apache.org>
Subject: Time difference between message fetch and message send is very high 
(~900-1200ms)


Hello All,

In my earlier mail, I am sorry I did not give lot of details on my setup
where I am measuring performance between producer and  consumer. Here are
the details:

- I am using Kafka version 0.7 downloaded from Apache web site.
- I am using a modified version of KafkaConsumerProducerDemo example that
comes with  the distribution. I have made the following modifications:

- In Producer java file:

    int messageNo = 1;
    SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd
HH:mm:ss.SSS") ;

    while(true)
    {
      String messageStr = new String("Message_" + messageNo);
      producer.send(new ProducerData<Integer, String>(topic,
        "Message produced at " + dateFormat.format(new Date())
        + ": " + messageStr));
      messageNo++;
    }


- In Consumer.java file:

    SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd
HH:mm:ss.SSS") ;

    while(it.hasNext())
      System.out.println("Message received at " + dateFormat.format(new
Date()) + ": " + ExampleUtils.getMessage(it.next().message()));


Based on the messages on the console from consumer, I am getting values
ranging from 900ms to 1200ms per message for the time difference between
received message time and the timestamp in the message itself.

Kindly let me know where I am going wrong and are there any configuration
steps involved for getting optimal performance for the time taken for a
specific message to be fetched and the time when it was sent by producer.

Regards,
Kishore