[
https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14145096#comment-14145096
]
Joe Stein edited comment on KAFKA-1555 at 9/23/14 5:39 PM:
-----------------------------------------------------------
[~gwenshap] yup, in that case the issue was between the chair and the keyboard
(me)
{code}
root@precise64:/opt/apache/kafka# bin/kafka-console-producer.sh --broker-list
localhost:9092 --topic testNew --sync --request-required-acks -1
A
[2014-09-23 17:36:37,127] WARN Produce request with correlation id 2 failed due
to [testNew,1]: kafka.common.NotEnoughReplicasException
(kafka.producer.async.DefaultEventHandler)
[2014-09-23 17:36:37,248] WARN Produce request with correlation id 5 failed due
to [testNew,2]: kafka.common.NotEnoughReplicasException
(kafka.producer.async.DefaultEventHandler)
[2014-09-23 17:36:37,364] WARN Produce request with correlation id 8 failed due
to [testNew,2]: kafka.common.NotEnoughReplicasException
(kafka.producer.async.DefaultEventHandler)
[2014-09-23 17:36:37,480] WARN Produce request with correlation id 11 failed
due to [testNew,1]: kafka.common.NotEnoughReplicasException
(kafka.producer.async.DefaultEventHandler)
[2014-09-23 17:36:37,591] ERROR Failed to send requests for topics testNew with
correlation ids in [0,12] (kafka.producer.async.DefaultEventHandler)
kafka.common.FailedToSendMessageException: Failed to send messages after 3
tries.
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at kafka.producer.Producer.send(Producer.scala:76)
at kafka.producer.OldProducer.send(BaseProducer.scala:62)
at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:95)
at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
{code}
It is working, awesome!
was (Author: joestein):
[~gwenshap] yup, in that case the issue was between the chair and the keyboard
(me)
{code}
root@precise64:/opt/apache/kafka# bin/kafka-console-producer.sh --broker-list
localhost:9092 --topic testNew --sync --request-required-acks -1
A
[2014-09-23 17:36:37,127] WARN Produce request with correlation id 2 failed due
to [testNew,1]: kafka.common.NotEnoughReplicasException
(kafka.producer.async.DefaultEventHandler)
[2014-09-23 17:36:37,248] WARN Produce request with correlation id 5 failed due
to [testNew,2]: kafka.common.NotEnoughReplicasException
(kafka.producer.async.DefaultEventHandler)
[2014-09-23 17:36:37,364] WARN Produce request with correlation id 8 failed due
to [testNew,2]: kafka.common.NotEnoughReplicasException
(kafka.producer.async.DefaultEventHandler)
[2014-09-23 17:36:37,480] WARN Produce request with correlation id 11 failed
due to [testNew,1]: kafka.common.NotEnoughReplicasException
(kafka.producer.async.DefaultEventHandler)
[2014-09-23 17:36:37,591] ERROR Failed to send requests for topics testNew with
correlation ids in [0,12] (kafka.producer.async.DefaultEventHandler)
kafka.common.FailedToSendMessageException: Failed to send messages after 3
tries.
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
at kafka.producer.Producer.send(Producer.scala:76)
at kafka.producer.OldProducer.send(BaseProducer.scala:62)
at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:95)
at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
{code}
awesome!
> provide strong consistency with reasonable availability
> -------------------------------------------------------
>
> Key: KAFKA-1555
> URL: https://issues.apache.org/jira/browse/KAFKA-1555
> Project: Kafka
> Issue Type: Improvement
> Components: controller
> Affects Versions: 0.8.1.1
> Reporter: Jiang Wu
> Assignee: Gwen Shapira
> Fix For: 0.8.2
>
> Attachments: KAFKA-1555.0.patch, KAFKA-1555.1.patch,
> KAFKA-1555.2.patch, KAFKA-1555.3.patch
>
>
> In a mission critical application, we expect a kafka cluster with 3 brokers
> can satisfy two requirements:
> 1. When 1 broker is down, no message loss or service blocking happens.
> 2. In worse cases such as two brokers are down, service can be blocked, but
> no message loss happens.
> We found that current kafka versoin (0.8.1.1) cannot achieve the requirements
> due to its three behaviors:
> 1. when choosing a new leader from 2 followers in ISR, the one with less
> messages may be chosen as the leader.
> 2. even when replica.lag.max.messages=0, a follower can stay in ISR when it
> has less messages than the leader.
> 3. ISR can contains only 1 broker, therefore acknowledged messages may be
> stored in only 1 broker.
> The following is an analytical proof.
> We consider a cluster with 3 brokers and a topic with 3 replicas, and assume
> that at the beginning, all 3 replicas, leader A, followers B and C, are in
> sync, i.e., they have the same messages and are all in ISR.
> According to the value of request.required.acks (acks for short), there are
> the following cases.
> 1. acks=0, 1, 3. Obviously these settings do not satisfy the requirement.
> 2. acks=2. Producer sends a message m. It's acknowledged by A and B. At this
> time, although C hasn't received m, C is still in ISR. If A is killed, C can
> be elected as the new leader, and consumers will miss m.
> 3. acks=-1. B and C restart and are removed from ISR. Producer sends a
> message m to A, and receives an acknowledgement. Disk failure happens in A
> before B and C replicate m. Message m is lost.
> In summary, any existing configuration cannot satisfy the requirements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
