Hi,

I am facing an issue with the Kafka clustering setup that I have. I have a 
Kafka cluster with two broker that are connected to two zookeepers. I am 
posting data to a topic that have replication factor and partition two each 
with a spring boot Kafka producer and consuming the same with another spring 
boot app.

I found one strange behavior when testing the cluster in the following manner -

  1.   Turned off node1 and node 2
  2.   Turned on node 1
  3.   Turned off node 1
  4.   Turned on node 2

After turning on node 2 Kafka cluster got failed and I am not able to produce 
data to Kafka. My consumer started throwing the message continuously as given 
below.

 [Producer clientId=producer-1] Connection to node 1 (/server1-ip:9092) could 
not be established. Broker may not be available.

Issue is visible in both nodes. But if I kept both system up for a while issue 
will get resolved and I can turn off any of the node without breaking the 
cluster.
My broker configuration is as below.

broker.id=0
listeners=PLAINTEXT://server1-ip:9092
advertised.listeners=PLAINTEXT://serever1-ip:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/home/user/kafka/data/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=2
offsets.topic.replication.factor=2
transaction.state.log.replication.factor=2
transaction.state.log.min.isr=2
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=serever1-ip:2181,serever2-ip:2181
zookeeper.connection.timeout.ms=6000
group.initial.rebalance.delay.ms=3000
auto.leader.rebalance.enable=true
leader.imbalance.check.interval.seconds=5

Zookeeper configuration

dataDir=/home/user/kafka/data
clientPort=2181
maxClientCnxns=0
initLimit=10
syncLimit=5
tickTime=2000
server.1=server1-ip:2888:3888
server.2=server2-ip:2888:3888

Is this is an expected behavior of Kafka or am I doing something wrong with 
this configuration ?

Can somebody help me with this issue ..

Thanks in advance.


Reply via email to