[
https://issues.apache.org/jira/browse/KAFKA-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14142967#comment-14142967
]
Sriram Subramanian commented on KAFKA-1555:
-------------------------------------------
Sorry to be late here but I think this an important change and we need to
ensure this is perfectly the right behavior for the long term.
To summarize the discussion and code change so far -
1. We would set the min.isr per topic in log config
2. We would use this config only when ack is set to -1 and fail the call if the
number of in sync replicas is less than min isr
The main drawbacks I see with this approach are -
1. If we plan to set this value at a per topic level, this should be part of
create/modify topic and should be set during topic creation or modified later.
This ensures that if we do expose a createTopic api in the protocol, it would
be available to be set/modified.
2. I could see scenarios where multiple writers could have different
requirements on the same topic and may not have any knowledge of how the topic
was created.
3. I think what we are really solving for is to either make the write durable
on all replicas or on just the in sync replicas. The min.isr value provides the
option of a number and I think any value other than 0 or no_of_replicas is of
no value. This would only confuse the clients when they create the topic.
This is how I interpret the acks w.r.t the clients -
0 - No response required. I don't really care if the write happened
1 - I need a response after the write happened to the leader successfully
> 1 - I need the write to happen on all replicas before a response. This has
> two options -
a. Response is sent after write happens to replicas in ISR
b. Response is sent after write happens to all replicas
Having an enum for ack as below is a lot clearer and sets the expectations
right in my opinion.
enum AckType {
No_Response,
Write_To_Leader,
Write_To_ISR, (Chooses availability over consistency)
Write_To_All_Replicas (Chooses consistency over availability)
}
> provide strong consistency with reasonable availability
> -------------------------------------------------------
>
> Key: KAFKA-1555
> URL: https://issues.apache.org/jira/browse/KAFKA-1555
> Project: Kafka
> Issue Type: Improvement
> Components: controller
> Affects Versions: 0.8.1.1
> Reporter: Jiang Wu
> Assignee: Gwen Shapira
> Fix For: 0.8.2
>
> Attachments: KAFKA-1555.0.patch, KAFKA-1555.1.patch
>
>
> In a mission critical application, we expect a kafka cluster with 3 brokers
> can satisfy two requirements:
> 1. When 1 broker is down, no message loss or service blocking happens.
> 2. In worse cases such as two brokers are down, service can be blocked, but
> no message loss happens.
> We found that current kafka versoin (0.8.1.1) cannot achieve the requirements
> due to its three behaviors:
> 1. when choosing a new leader from 2 followers in ISR, the one with less
> messages may be chosen as the leader.
> 2. even when replica.lag.max.messages=0, a follower can stay in ISR when it
> has less messages than the leader.
> 3. ISR can contains only 1 broker, therefore acknowledged messages may be
> stored in only 1 broker.
> The following is an analytical proof.
> We consider a cluster with 3 brokers and a topic with 3 replicas, and assume
> that at the beginning, all 3 replicas, leader A, followers B and C, are in
> sync, i.e., they have the same messages and are all in ISR.
> According to the value of request.required.acks (acks for short), there are
> the following cases.
> 1. acks=0, 1, 3. Obviously these settings do not satisfy the requirement.
> 2. acks=2. Producer sends a message m. It's acknowledged by A and B. At this
> time, although C hasn't received m, C is still in ISR. If A is killed, C can
> be elected as the new leader, and consumers will miss m.
> 3. acks=-1. B and C restart and are removed from ISR. Producer sends a
> message m to A, and receives an acknowledgement. Disk failure happens in A
> before B and C replicate m. Message m is lost.
> In summary, any existing configuration cannot satisfy the requirements.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
