Hi, I have been struggling for this connection problem for a whole week. I run Kafka on my server machine which is on Cloud. And I cannot manage to connect to Kafka in anyway. It always print out `Connection to node -1 (myServerDomain.ltd/myServerIp:9092) could not be established. Broker may not be available.`
Here is what I had done to keep it simple so I can find out why: - I stopped using Docker. - I configure the Cloud firewall, so it will not block port 9092. - I start Kafka standalone(1 broker). - I start zookeeper standalone(1 node). - Zookeeper and Kafka use JAAS to connect. - I configure Kafka to log in TRACE level. - SASL and SSL is all turn off. I know advertised.listeners is usually the key of these situation, so here is what I had tried: SITUATION 1: - set advertised.listener to `localhost` on propose - use `kafka-topic --list --bootstrap-server localhost:9092`: 1. Kafka print out the metadata request 2. Kafka print out the returned metadata 3. Client connect to the advertised listener(which is localhost) successfully 4. Topics list is returned, client print out topic list. All well. - use `kafka-topic --list --bootstrap-server xxx.ltd:9092` 1. Kafka even not print out the first metadata request. 2. In client, it print out(Notice, it is node -1, not node 1): `Connection to node -1 (myServerDomain.ltd/myServerIp:9092) could not be established. Broker may not be available.` 3. Stop Kafka, start a WWW service on port 9092, can access the WWW service by port 9092. SITUATION 2: - set advertised.listener to `xxx.ltd` - use `kafka-topic --list --bootstrap-server localhost:9092`: 1. Kafka print out the metadata request 2. Kafka print out the returned metadata 3. Client try to connect to the advertised listener(which is xxx.ltd) 4. Failed, it print out(Notice,it is node 1, not node -1 like above, that means, client is try to connect by the metadata returned by kafka): `Connection to node 1 (myServerDomain.ltd/myServerIp:9092) could not be established. Broker may not be available.` - use `kafka-topic --list --bootstrap-server xxx.ltd:9092`: 1. Kafka even not print out the first metadat request. 2. In client, it print out(Notice is node -1, not node 1): `Connection to node -1 (myServerDomain.ltd/myServerIp:9092) could not be established. Broker may not be available.` So, maybe, I think, maybe there is not any TCP request, all request to myServerDomain.ltd/myServerIp:9092 is blocked somehow. So I use `tcpdump -p 9092` to capture packets, here is what I get: --------------------------------------------- ... 2020-05-19 17:34:41.428139 IP 172.18.118.28.9092 > 61.140.182.143.5826: Flags [R.], seq 0, ack 4281665850, win 0, length 0 2020-05-19 17:34:41.842286 IP 61.140.182.143.5828 > 172.18.118.28.9092: Flags [S], seq 3141006320, win 64240, options [mss 1400,sackOK,TS val 1788286298 ecr 0,nop,wscale 1], length 0 2020-05-19 17:34:41.842360 IP 172.18.118.28.9092 > 61.140.182.143.5828: Flags [R.], seq 0, ack 3141006321, win 0, length 0 2020-05-19 17:34:42.657551 IP 61.140.182.143.5833 > 172.18.118.28.9092: Flags [S], seq 44626980, win 64240, options [mss 1400,sackOK,TS val 1788287114 ecr 0,nop,wscale 1], length 0 2020-05-19 17:34:42.657604 IP 172.18.118.28.9092 > 61.140.182.143.5833: Flags [R.], seq 0, ack 44626981, win 0, length ... 61.140.182.143 is my local laptop. It seems they were communicating. This is my `server.properties` config: -------------------------------------------------------- broker.id=1 num.network.threads=3 num.io.threads=8 socket.send.buffer.bytes=102400 socket.receive.buffer.bytes=102400 socket.request.max.bytes=104857600 log.dirs=/var/log/kafka-1 num.partitions=3 num.recovery.threads.per.data.dir=3 offset.topic.replication.factor=3 transaction.state.log.replication.factor=3 transaction.state.log.min.isr=3 zookeeper.connect=zookeeper1.myServerDomain.ltd:2181(Original set for Docker network, now I use /etc/hosts map to localhost) zookeeper.connection.timeout.ms=18000 listener.security.protocol.map=INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT listeners=INSIDE://kafka1.myServerDomain.ltd:19092,OUTSIDE://localhost:9092 advertised.listeners=INSIDE://kafka1.myServerDomain.ltd:19092,OUTSIDE://myServerDomain.ltd:9092 inter.broker.listener.name=INSIDE -------------------------------------------------------- I am almost desperate, ANY SOLUTION OR DIRECTION WOULD BE VERY APPRECIATED!!!
