Hi Daniele, Is it possible that the IP addresses of your brokers change after you start your server instances again, and the producers can't connect because the new IPs are not in their configuration? That could also be an explanation for why the new broker is in the ISR list - I think the brokers learn about each other through zookeeper and a different IP wouldn't matter.
Harper On Fri, Jan 25, 2019 at 6:10 AM Daniele Bosetti <d.bose...@cmcmarkets.com> wrote: > Hi, > > I am evaluating Kafka, and I am testing failover of kafka servers. > > My problem is, if a server comes back online after being turned off, the > client seems not to notice it. > > The setup is: > 3 zookeeper instances on boxes A,B,C > 2 kafka server instances on boxes A,B > 1 client running on the dev machine > I am using the java library, org.apache.kafka:kafka-clients:2.1.0 > > The client-producer sends messages every second, to a test topic with > replication factor=2 and partitionCount=1 > The leader is broker-0 (on box A) ; when I shut down that server instance, > broker-1 becomes the leader. > The client reports the disconnection from broker-0 and happily connects to > broker-1, messages are sent correctly. > > Then I turn broker-0 back on, and check that this instance is back in the > replica set using kafka-topics.sh (output is below). > But then, when I turn off broker-1 , the client does not reconnect to > broker-0; > And I have errors > 2019-01-25 13:46:18,777 [kafka-producer-network-thread | producer-1] DEBUG > org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] > Give up sending metadata request since no node is available > 2019-01-25 13:46:18,828 [kafka-producer-network-thread | producer-1] DEBUG > org.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] > Give up sending metadata request since no node is available > > > [..]$ bin/kafka-topics.sh --describe --zookeeper boxA:12181 --topic > test-topic-3 > Topic:test-topic-3 PartitionCount:1 ReplicationFactor:2 > Configs: > Topic: test-topic-3 Partition: 0 Leader: 1 Replicas: > 1,0 Isr: 1,0 > > > I am a bit of a loss, I am not sure if this is a configuration problem? > So my questions are, > (1) in a correct setup, the client should automatically reconnect to the > available broker after it comes back up? > (2) how can I fix/troubleshoot my setup? > (3) we don't need additional connection-listeners, or cluster-listeners, > on the client side (right?) > > > Any help is appreciated! > Thanks, > Daniele > ________________________________ > > > Spread bets and CFDs are complex instruments and come with a high risk of > losing money rapidly due to leverage. 80% of retail investor accounts lose > money when spread betting and/or trading CFDs with this provider. You > should consider whether you understand how spread bets and CFDs work and > whether you can afford to take the high risk of losing your money. > Professional clients: Digital 100s and Countdowns carry a level of risk to > your capital as you could lose all of your investment. These products may > not be suitable for all clients therefore ensure you understand the risks > and seek independent advice. Invest only what you can afford to lose. > > CMC Markets UK plc (173730) and CMC Spreadbet plc (170627) are authorised > and regulated by the Financial Conduct Authority in the United Kingdom. CMC > Markets UK plc and CMC Spreadbet plc are registered in England and Wales > with Company Numbers 02448409 and 02589529 and with their registered > offices at 133 Houndsditch, London, EC3A 7BX. > > The content of this e-mail (including any attachments) is strictly > confidential and is for the sole use of the intended addressee(s). If you > are not the intended recipient of this e-mail please notify the sender > immediately and delete this e-mail from your system. Any disclosure, > copying, dissemination or use of its content (including any attachments) is > strictly prohibited. CMC Markets UK plc and CMC Spreadbet plc reserve the > right to intercept and monitor the content of the e-mail messages to and > from its systems. > > E-mails may be interfered with or may contain viruses or other defects for > which CMC Markets UK plc and CMC Spreadbet plc accept no responsibility. It > is the responsibility of the recipient to carry out a virus check on the > e-mail and any attachment(s). > > This communication is not intended as an offer or solicitation for the > purchase or sale of a financial instrument or as an official confirmation > of any transaction unless specifically presented as such. > > CMC Markets UK plc and CMC Spreadbet plc are registered in England and > Wales with Company Numbers 02448409 and 02589529 and with their registered > offices at 133 Houndsditch, London, EC3A 7BX. >
