Kishore,
The end to end latency will be much lower in 08. For 07, you can try to
tune the producer batch size so it sends data quicker. You can also reduce
the log.flush.interval on the broker but that will drive up iops.
Thanks,
Neha
On May 16, 2013 11:39 PM, "Kishore V. Kopalle" <kish...@greenmedsoft.com>
wrote:
> Hi Frank,
>
> Here are the explicitly set properties:
> Consumer:
> Properties props = new Properties();
> props.put("zk.connect", KafkaProperties.zkConnect);
> props.put("groupid", KafkaProperties.groupId);
> props.put("zk.sessiontimeout.ms", "400");
> props.put("zk.synctime.ms", "200");
> props.put("autocommit.interval.ms", "1000");
>
> Producer:
> props.put("serializer.class", "kafka.serializer.StringEncoder");
> props.put("zk.connect", "localhost:2181");
>
> Everything else is out of box except for log.default.flush.interval.ms=1
> in
> server.properties which is set according to Stone's suggestion.
>
>
> Regards,
> Kishore
>
> On Fri, May 17, 2013 at 11:48 AM, Francis Dallaire <
> francis.dalla...@ubisoft.com> wrote:
>
> > What are the settings you have set on the producer ... is there any sync
> > ack ? If so which type ?
> >
> > In any case, it would be useful to know the parameters you set on the
> > consumer and producer.
> >
> > Thanks
> >
> > Frank
> >
> >
> > Sent from Samsung Mobile
> >
> >
> >
> > -------- Original message --------
> > From: "Kishore V. Kopalle" <kish...@greenmedsoft.com>
> > Date: 05-17-2013 2:11 AM (GMT-05:00)
> > To: users@kafka.apache.org,Francis Dallaire <
> francis.dalla...@ubisoft.com>
> > Subject: Re: Time difference between message fetch and message send is
> > very high (~900-1200ms)
> >
> >
> > Hi Frank,
> >
> > Yes, they are on the same machine. Zoo Keeper, Kafka server and the
> > KafkaConsumerProducerDemo (with one producer and one consumer) are
> running
> > on the same computer.
> >
> > Regards,
> > Kishore
> >
> > On Fri, May 17, 2013 at 11:38 AM, Francis Dallaire <
> > francis.dalla...@ubisoft.com<mailto:francis.dalla...@ubisoft.com>>
> wrote:
> > Quick question. Are the producer and consumer on the same machine ? If
> > not, are you accounting for any difference in the time setting between
> > producer and consumer ?
> >
> > Frank
> >
> > Sent from Samsung Mobile
> >
> >
> >
> > -------- Original message --------
> > From: Kishore Kopalle <kkopa...@gmail.com<mailto:kkopa...@gmail.com>>
> > Date: 05-17-2013 1:41 AM (GMT-05:00)
> > To: users@kafka.apache.org<mailto:users@kafka.apache.org>
> > Subject: Time difference between message fetch and message send is very
> > high (~900-1200ms)
> >
> >
> > Hello All,
> >
> > In my earlier mail, I am sorry I did not give lot of details on my setup
> > where I am measuring performance between producer and consumer. Here are
> > the details:
> >
> > - I am using Kafka version 0.7 downloaded from Apache web site.
> > - I am using a modified version of KafkaConsumerProducerDemo example that
> > comes with the distribution. I have made the following modifications:
> >
> > - In Producer java file:
> >
> > int messageNo = 1;
> > SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd
> > HH:mm:ss.SSS") ;
> >
> > while(true)
> > {
> > String messageStr = new String("Message_" + messageNo);
> > producer.send(new ProducerData<Integer, String>(topic,
> > "Message produced at " + dateFormat.format(new Date())
> > + ": " + messageStr));
> > messageNo++;
> > }
> >
> >
> > - In Consumer.java file:
> >
> > SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd
> > HH:mm:ss.SSS") ;
> >
> > while(it.hasNext())
> > System.out.println("Message received at " + dateFormat.format(new
> > Date()) + ": " + ExampleUtils.getMessage(it.next().message()));
> >
> >
> > Based on the messages on the console from consumer, I am getting values
> > ranging from 900ms to 1200ms per message for the time difference between
> > received message time and the timestamp in the message itself.
> >
> > Kindly let me know where I am going wrong and are there any configuration
> > steps involved for getting optimal performance for the time taken for a
> > specific message to be fetched and the time when it was sent by producer.
> >
> > Regards,
> > Kishore
> >
> >
>
