Hi,
I am evaluating Kafka, and I am testing failover of kafka servers.
My problem is, if a server comes back online after being turned off, the client
seems not to notice it.
The setup is:
3 zookeeper instances on boxes A,B,C
2 kafka server instances on boxes A,B
1 client running on the dev machine
I am using the java library, org.apache.kafka:kafka-clients:2.1.0
The client-producer sends messages every second, to a test topic with
replication factor=2 and partitionCount=1
The leader is broker-0 (on box A) ; when I shut down that server instance,
broker-1 becomes the leader.
The client reports the disconnection from broker-0 and happily connects to
broker-1, messages are sent correctly.
Then I turn broker-0 back on, and check that this instance is back in the
replica set using kafka-topics.sh (output is below).
But then, when I turn off broker-1 , the client does not reconnect to broker-0;
And I have errors
2019-01-25 13:46:18,777 [kafka-producer-network-thread | producer-1] DEBUG
org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up
sending metadata request since no node is available
2019-01-25 13:46:18,828 [kafka-producer-network-thread | producer-1] DEBUG
org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Give up
sending metadata request since no node is available
[..]$ bin/kafka-topics.sh --describe --zookeeper boxA:12181 --topic test-topic-3
Topic:test-topic-3 PartitionCount:1 ReplicationFactor:2 Configs:
Topic: test-topic-3 Partition: 0 Leader: 1 Replicas: 1,0
Isr: 1,0
I am a bit of a loss, I am not sure if this is a configuration problem?
So my questions are,
(1) in a correct setup, the client should automatically reconnect to the
available broker after it comes back up?
(2) how can I fix/troubleshoot my setup?
(3) we don't need additional connection-listeners, or cluster-listeners, on the
client side (right?)
Any help is appreciated!
Thanks,
Daniele
________________________________
Spread bets and CFDs are complex instruments and come with a high risk of
losing money rapidly due to leverage. 80% of retail investor accounts lose
money when spread betting and/or trading CFDs with this provider. You should
consider whether you understand how spread bets and CFDs work and whether you
can afford to take the high risk of losing your money. Professional clients:
Digital 100s and Countdowns carry a level of risk to your capital as you could
lose all of your investment. These products may not be suitable for all clients
therefore ensure you understand the risks and seek independent advice. Invest
only what you can afford to lose.
CMC Markets UK plc (173730) and CMC Spreadbet plc (170627) are authorised and
regulated by the Financial Conduct Authority in the United Kingdom. CMC Markets
UK plc and CMC Spreadbet plc are registered in England and Wales with Company
Numbers 02448409 and 02589529 and with their registered offices at 133
Houndsditch, London, EC3A 7BX.
The content of this e-mail (including any attachments) is strictly confidential
and is for the sole use of the intended addressee(s). If you are not the
intended recipient of this e-mail please notify the sender immediately and
delete this e-mail from your system. Any disclosure, copying, dissemination or
use of its content (including any attachments) is strictly prohibited. CMC
Markets UK plc and CMC Spreadbet plc reserve the right to intercept and monitor
the content of the e-mail messages to and from its systems.
E-mails may be interfered with or may contain viruses or other defects for
which CMC Markets UK plc and CMC Spreadbet plc accept no responsibility. It is
the responsibility of the recipient to carry out a virus check on the e-mail
and any attachment(s).
This communication is not intended as an offer or solicitation for the purchase
or sale of a financial instrument or as an official confirmation of any
transaction unless specifically presented as such.
CMC Markets UK plc and CMC Spreadbet plc are registered in England and Wales
with Company Numbers 02448409 and 02589529 and with their registered offices at
133 Houndsditch, London, EC3A 7BX.
