Hi,
Just to confirm that the number of messages produced are what you are
seeing, What does GetOffsetShell report for this topic ?
Regards,
Prabhjot
On Thu, Nov 12, 2015 at 2:13 PM, Hawin Jiang <hawin.ji...@gmail.com> wrote:
> Hi Jinxing
>
> I don't think we can resolve this issue by increasing producers. if I
> increased more producers, it should lost more messages.
>
> I just test two producers.
> Thread Producer 1 has 83064 messages in producer side and 82273 messages in
> consumer side
> Thread Producer 2 has 89844 messages in producer side and 88892 messages in
> consumer side.
>
> Thanks.
>
>
>
> Best regards
> Hawin
>
>
> On Thu, Nov 12, 2015 at 12:24 AM, jinxing <jinxing6...@126.com> wrote:
>
> > maybe there some changes in 0.9.0.0;
> >
> >
> > but still you can try increase producer sending rate, and see if there
> are
> > message lost but no exception;
> >
> >
> > note that, to increase the producer sending rate, you must have enough
> > producer 'power';
> >
> >
> > in my case, I have 50 producer sending message at the same time : )
> >
> >
> >
> >
> >
> >
> >
> > At 2015-11-12 16:16:23, "Hawin Jiang" <hawin.ji...@gmail.com> wrote:
> > >Hi Jinxing
> > >
> > >I am using kafka_2.10-0.9.0.0-SNAPSHOT. I have downloaded source code
> and
> > >installed it last week.
> > >
> > >I saw 97446 messages have been sent to kafka successfully.
> > >
> > >So far, I have not found any failed messages.
> > >
> > >
> > >
> > >Best regards
> > >Hawin
> > >
> > >On Thu, Nov 12, 2015 at 12:07 AM, jinxing <jinxing6...@126.com> wrote:
> > >
> > >> Hi, what version are you using ?
> > >>
> > >>
> > >> i am using 0.8.2.0, and I encountered this problem before;
> > >>
> > >>
> > >> it seems that if the message rate of the producer side is to high,
> some
> > of
> > >> the messages will lost;
> > >>
> > >>
> > >> also, i found that the callback method of the producer 'send' API is
> not
> > >> reliable;
> > >>
> > >>
> > >> only successful sent message will trigger the callback, but the failed
> > >> ones don't;
> > >>
> > >>
> > >> you saw this?
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> At 2015-11-12 16:01:17, "Hawin Jiang" <hawin.ji...@gmail.com> wrote:
> > >> >Hi All
> > >> >
> > >> >I have sent messages to Kafka for one minute. I found 97446 messages
> > in
> > >> >producer side and 96896 messages in consumer side for Case 1.
> > >> >I also tried case 2. I have faced the same issues. The number is not
> > >> match
> > >> >between producer and consumer.
> > >> >Can someone take a look at this issue?
> > >> >Thanks.
> > >> >
> > >> >
> > >> >Case 1:
> > >> >
> > >> >long startTime = System.currentTimeMillis();
> > >> >long maxDurationInMilliseconds = 1 * 60 * 1000;
> > >> >int messageNo = 0;
> > >> >while (true) {
> > >> >if (System.currentTimeMillis() <= startTime
> > >> >+ maxDurationInMilliseconds) {
> > >> >messageNo++;
> > >> >String messageStr = "KAFKA_"+messageNo;
> > >> >System.out.println("Message: "+messageNo);
> > >> >producer.send(new KeyedMessage<Integer, String>(topic,messageStr));
> > >> >} else {
> > >> >producer.close();
> > >> >System.out.println("Total kafka Message: "+messageNo);
> > >> >break;
> > >> >}
> > >> >}
> > >> >
> > >> >
> > >> >Case 2:
> > >> >
> > >> >for (int i=1;i<=12000;i++)
> > >> >String messageStr = "KAFKA_"+i;
> > >> >producer.send(new KeyedMessage<Integer, String>(topic,messageStr));
> > >> >
> > >> >
> > >> >
> > >> >Best regards
> > >> >Hawin
> > >>
> >
>
--
---------------------------------------------------------
"There are only 10 types of people in the world: Those who understand
binary, and those who don't"
