Re: kafka broker is dropping the messages after acknowledging librdkafka

2016-08-17 Thread Mazhar Shaikh
. > > I'm running kafka broker with replication factor of 2. > > > > Here, message has been lost between librdkafka - kafka broker. > > > > As librdkafka is providing success delivery report for all the messages. > > > > Looks like kafka broker is droppin

Re: kafka broker is dropping the messages after acknowledging librdkafka

2016-08-17 Thread Jun Rao
een lost between librdkafka - kafka broker. > > As librdkafka is providing success delivery report for all the messages. > > Looks like kafka broker is dropping the messages after acknowledging > librdkafka. > > Requesting you help in solving this issue. > > Thank you. > > > Regards > Mazhar Shaikh >

Re: kafka broker is dropping the messages after acknowledging librdkafka

2016-08-17 Thread Mazhar Shaikh
geNotificationListener) [2016-08-17 13:10:43,383] DEBUG Sending MetadataRequest to Brokers:ArrayBuffer(0, 5, 1, 2, 3, 4) for TopicAndPartitions:Set([topic1,67], [topic1,95]) (kafka.controller.IsrChangeNotificationListener) [2016-08-17 13:10:43,394] DEBUG [IsrChangeNotificationListener] Fired!!! (kafka.controller.IsrChangeNotificationListener) Thanks Regards, Mazhar Shaikh. On Thu, Aug 11, 2016 at 7:03 PM, Tom Crayford wrote: > Are you running with unclean leader election on? Are you setting min in > sync replicas at all? > > Can you attach controller and any other logs from the brokers you have? > They would be crucial in debugging this kind of issue. > > Thanks > > Tom Crayford > Heroku Kafka > > On Thursday, 11 August 2016, Mazhar Shaikh > wrote: > > > Hi Kafka Team, > > > > I'm using kafka (kafka_2.11-0.9.0.1) with librdkafka (0.8.1) API for > > producer > > During a run of 2hrs, I notice the total number of messaged ack'd by > > librdkafka delivery report is greater than the maxoffset of a partition > in > > kafka broker. > > I'm running kafka broker with replication factor of 2. > > > > Here, message has been lost between librdkafka - kafka broker. > > > > As librdkafka is providing success delivery report for all the messages. > > > > Looks like kafka broker is dropping the messages after acknowledging > > librdkafka. > > > > Requesting you help in solving this issue. > > > > Thank you. > > > > > > Regards > > Mazhar Shaikh > > >

kafka broker is dropping the messages after acknowledging librdkafka

2016-08-11 Thread Mazhar Shaikh
ith replication factor of 2. Here, message has been lost between librdkafka - kafka broker. As librdkafka is providing success delivery report for all the messages. Looks like kafka broker is dropping the messages after acknowledging librdkafka. Requesting you help in solving this issue. Thank

Re: kafka broker is dropping the messages after acknowledging librdkafka

2016-08-11 Thread Tom Crayford
fka broker. > I'm running kafka broker with replication factor of 2. > > Here, message has been lost between librdkafka - kafka broker. > > As librdkafka is providing success delivery report for all the messages. > > Looks like kafka broker is dropping the messages after ackno