I've have some questions regarding Kafka ISR's and the number of servers one 
should have.


In our production environment we have a 6 node Kafka cluster with 
min.insync.replicas=2, and

all of our topics have a ReplicationFactor: of 2


When we create topics,  we do not pin them to a specific broker.  A 10 
partition looks like this:


Topic:<FOO>    PartitionCount:10    ReplicationFactor:2    Configs:

    Topic: <FOO>    Partition: 0    Leader: 6    Replicas: 6,5    Isr: 6,5
    Topic: <FOO>    Partition: 1    Leader: 1    Replicas: 1,6    Isr: 1,6
    Topic: <FOO>    Partition: 2    Leader: 2    Replicas: 2,1    Isr: 1,2
    Topic: <FOO>    Partition: 3    Leader: 3    Replicas: 3,2    Isr: 3,2
    Topic: <FOO>    Partition: 4    Leader: 4    Replicas: 4,3    Isr: 4,3
    Topic: <FOO>    Partition: 5    Leader: 5    Replicas: 5,4    Isr: 5,4
    Topic: <FOO>    Partition: 6    Leader: 6    Replicas: 6,1    Isr: 1,6
    Topic: <FOO>    Partition: 7    Leader: 1    Replicas: 1,2    Isr: 1,2
    Topic: <FOO>    Partition: 8    Leader: 2    Replicas: 2,3    Isr: 3,2
    Topic: <FOO>    Partition: 9    Leader: 3    Replicas: 3,4    Isr: 4,3



Whenever we lose 1 node,  the cluster will not allow anyone to produce and the 
entire cluster fills our logs with:


The size of the current ISR Set(1) is insufficient to satisfy the min.isr 
requirement of 2 for partition <FOO> .





Do I need more brokers to sustain losing 1 ?


Is my partition schema out of whack ?


Is the a formula out there that describes the number of brokers to isr's to 
replicas  ?


Should I tweak the replica.fetch.wait.max.ms ?



Any guidance will be highly appreciated.


--shamer


Reply via email to