Hi Jun / Kafka Team,
Do we have any solution for this issue ?
"During zookeeper re-init kafka broker truncates messages and ends up in
loosing records"
I'm ok with with duplicate messages being stored instead of dropping.
Is there any configuration in kafka where follower broker replicates thes
Hi Jun,
Yes, the data is lost during leader broker failure.
But the leader broker failed due to zookeeper session expiry.
GC logs doesn't show any error/warns during this period.
Its not easy reproduce. during long run (>12hrs) with 30k msg/sec load
balanced across 96 partitions, some time in bet
Mazhar,
Let's first confirm if this is indeed a bug. As I mentioned earlier, it's
possible to have message loss with ack=1 when there are (leader) broker
failures. If this is not the case, please file a jira and describe how to
reproduce the problem. Also, it would be useful to know if the message
Hi Jun,
In my earlier runs, I had enabled delivery report (with and without offset
report) facility provided by librdkafka.
The producer has received successful delivery report for the all msg sent
even than the messages where lost.
as you mentioned. producer has nothing to do with this loss of
Mazhar,
With ack=1, whether you lose messages or not is not deterministic. It
depends on the time when the broker receives/acks a message, the follower
fetches the data and the broker fails. So, it's possible that you got lucky
in one version and unlucky in another.
Thanks,
Jun
On Thu, Aug 18,
Hi Jun,
Thanks for clarification, I'll give a try with ack=-1 (in producer).
However, i did a fallback to older version of kafka (*kafka_2.10-0.8.2.1*),
and i don't see this issue (loss of messages).
looks like kafka_2.11-0.9.0.1 has issues(BUG) during replication.
Thanks,
Regards,
Mazhar Shai
erred
> replica Map() (kafka.controller.KafkaController)
> [2016-08-17 13:09:50,296] TRACE [Controller 2]: leader imbalance ratio for
> broker 1 is 0.00 (kafka.controller.KafkaController)
> [2016-08-17 13:0
kaController)
> [2016-08-17 13:09:50,296] DEBUG [Controller 2]: topics not in preferred
> replica Map() (kafka.controller.KafkaController)
> [2016-08-17 13:09:50,296] TRACE [Controller 2]: leader imbalance ratio for
> broker 1 is 0.00 (kafka.controller.KafkaController)
> [20
troller.IsrChangeNotificationListener)
Thanks
Regards,
Mazhar Shaikh.
On Wed, Aug 17, 2016 at 9:50 PM, Jun Rao wrote:
> Yes, you can try setting it to -1 in 0.8.1, which is the equivalent of
> "all" in 0.9 and above.
>
> Thanks,
>
> Jun
>
> On Wed, Aug 17, 2016 at 8
; in
> > > kafka broker.
> > > I'm running kafka broker with replication factor of 2.
> > >
> > > Here, message has been lost between librdkafka - kafka broker.
> > >
> > > As librdkafka is providing success delivery report for all the
> messages.
> > >
> > > Looks like kafka broker is dropping the messages after acknowledging
> > > librdkafka.
> > >
> > > Requesting you help in solving this issue.
> > >
> > > Thank you.
> > >
> > >
> > > Regards
> > > Mazhar Shaikh
> > >
> >
>
.
> > I'm running kafka broker with replication factor of 2.
> >
> > Here, message has been lost between librdkafka - kafka broker.
> >
> > As librdkafka is providing success delivery report for all the messages.
> >
> > Looks like kafka broker is droppin
een lost between librdkafka - kafka broker.
>
> As librdkafka is providing success delivery report for all the messages.
>
> Looks like kafka broker is dropping the messages after acknowledging
> librdkafka.
>
> Requesting you help in solving this issue.
>
> Thank you.
>
>
> Regards
> Mazhar Shaikh
>
geNotificationListener)
[2016-08-17 13:10:43,383] DEBUG Sending MetadataRequest to
Brokers:ArrayBuffer(0, 5, 1, 2, 3, 4) for
TopicAndPartitions:Set([topic1,67], [topic1,95])
(kafka.controller.IsrChangeNotificationListener)
[2016-08-17 13:10:43,394] DEBUG [IsrChangeNotificationListener] Fired!!!
(kafka.controller.IsrChangeNotificationListener)
Thanks
Regards,
Mazhar Shaikh.
On Thu, Aug 11, 2016 at 7:03 PM, Tom Crayford wrote:
> Are you running with unclean leader election on? Are you setting min in
> sync replicas at all?
>
> Can you attach controller and any other logs from the brokers you have?
> They would be crucial in debugging this kind of issue.
>
> Thanks
>
> Tom Crayford
> Heroku Kafka
>
> On Thursday, 11 August 2016, Mazhar Shaikh
> wrote:
>
> > Hi Kafka Team,
> >
> > I'm using kafka (kafka_2.11-0.9.0.1) with librdkafka (0.8.1) API for
> > producer
> > During a run of 2hrs, I notice the total number of messaged ack'd by
> > librdkafka delivery report is greater than the maxoffset of a partition
> in
> > kafka broker.
> > I'm running kafka broker with replication factor of 2.
> >
> > Here, message has been lost between librdkafka - kafka broker.
> >
> > As librdkafka is providing success delivery report for all the messages.
> >
> > Looks like kafka broker is dropping the messages after acknowledging
> > librdkafka.
> >
> > Requesting you help in solving this issue.
> >
> > Thank you.
> >
> >
> > Regards
> > Mazhar Shaikh
> >
>
fka broker.
> I'm running kafka broker with replication factor of 2.
>
> Here, message has been lost between librdkafka - kafka broker.
>
> As librdkafka is providing success delivery report for all the messages.
>
> Looks like kafka broker is dropping the messages after ackno
ith replication factor of 2.
Here, message has been lost between librdkafka - kafka broker.
As librdkafka is providing success delivery report for all the messages.
Looks like kafka broker is dropping the messages after acknowledging
librdkafka.
Requesting you help in solving this issue.
Thank
15 matches
Mail list logo
