Client connects to IP address defined by bootstrap.servers and gets metada
that contain IP address where kafka is available. That is configured in
server.properties:
# Hostname and port the broker will advertise to producers and consumers. If
not set, # it uses the value for "listeners" if configured. Otherwise, it will
use the value# returned from
java.net.InetAddress.getCanonicalHostName().#advertised.listeners=PLAINTEXT://your.host.name:9092
you have to set that to 98.1.96.147:9092
advertised.listeners=PLAINTEXT://98,1,96.147:9092
Also you may need to change:
# The address the socket server listens on. It will get the value returned from
# java.net.InetAddress.getCanonicalHostName() if not configured.# FORMAT:#
listeners = listener_name://host_name:port# EXAMPLE:# listeners =
PLAINTEXT://your.host.name:9092#listeners=PLAINTEXT://:9092
listeners=PLAINTEXT://98,1,96.147:9092
Link to article:
Kafka Listeners - Explained
|
|
|
| | |
|
|
|
| |
Kafka Listeners - Explained
How to connect clients to Kafka hosted in separate networks, such as Docker,
AWS EC2, GCP, Azure, etc
|
|
|
On Tuesday, April 28, 2020, 07:04:42 PM GMT+2, Valentin Kulichenko
<valentin.kuliche...@gmail.com> wrote:
Hi Suresh,
Thanks for your quick response. All the configuration files, except for the
connect-standalone.properties are unmodified and use all the default
parameters. Attaching them anyway just in case.
This setup works perfectly on my laptop with VirtualBox (ZooKeeper and the
broker on VM, the connector on the host). So the issue seems to be dependable
on the environment.
Are there any particular scenarios that can cause Kafka Connect to try
localhost even if bootstrap.servers parameter provides a different IP? NAT?
Port forwarding? Something else?
I'm trying to get more details about the environment but would appreciate any
pointers in the meantime.
-Val
On Tue, Apr 28, 2020 at 12:07 AM Suresh Chidambaram <chida.sur...@gmail.com>
wrote:
Hi Val,
Could you share the server.properties and zookeeper.properties?
Thanks
C Suresh
On Tuesday, April 28, 2020, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:
> Greetings to the Kafka Community!
>
> I'm a newbie in Kafka and only recently went beyond a local installation
> described in the Quickstart. I have faced a weird issue that I can't
> explain.
>
> I want to deploy on two machines:
> - Machine #1 runs ZooKeeper and a single Kafka broker. I use default
> configuration files.
> - Machine #2 runs a single instance of Kafka Connect. I use the source
> connector for Ignite/GridGain (https://www.confluent.io/hub/
> gridgain/ignite-connector). I doubt it matters, though, as the issue
> seems to be related to connectivity between the connector and the broker.
>
> After starting the ZooKeeper and the broker on machine #1, I did the
> following on machine #2:
>
> 1. Updated plugin.path and bootstrap.servers parameters in the
> connect-standalone.properties; bootstrap.servers points to the IP of the #1.
> 2. Created a properties file for the GridGain connector.
> 3. Start the connector using this command:
>
> bin/connect-standalone.sh config/connect-standalone.properties
> config/gridgain-kafka-connect-source.properties
>
> The issue is that it doesn't connect to the remote broker, and still tries
> localhost ignoring the bootstrap.servers settings. The output looks like
> this:
>
> [2020-04-27 19:37:08,334] INFO StandaloneConfig values:
> ...
> bootstrap.servers = [98.1.96.147:9092]
> ...
> [2020-04-27 19:37:08,334] INFO Creating Kafka admin client
> (org.apache.kafka.connect.util.ConnectUtils:43)
> [2020-04-27 19:37:08,337] INFO AdminClientConfig values:
> bootstrap.servers = [98.1.96.147:9092]
> ...
> [2020-04-27 19:37:08,950] WARN [AdminClient clientId=adminclient-1]
> Connection to node 0 (localhost/127.0.0.1:9092) could not be established.
> Broker may not be available. (org.apache.kafka.clients.NetworkClient:756)
> [2020-04-27 19:37:09,052] WARN [AdminClient clientId=adminclient-1]
> Connection to node 0 (localhost/127.0.0.1:9092) could not be established.
> Broker may not be available. (org.apache.kafka.clients.NetworkClient:756)
> [2020-04-27 19:37:09,253] WARN [AdminClient clientId=adminclient-1]
> Connection to node 0 (localhost/127.0.0.1:9092) could not be established.
> Broker may not be available. (org.apache.kafka.clients.NetworkClient:756)
>
> The warning goes on for a while until the process stops after a timeout.
>
> Does anyone have any idea why Kafka Connect can try to establish a
> connection with the localhost, although it has picked up the correct IP
> from the configuration file?
>
> The issue doesn't seem to be reproducible on any environment, so I
> would appreciate any pointers on how to investigate this further.
>
> Kafka's version is 2.4.1. Config files and the log are attached. If
> anything else is needed, let me know.
>
> Regards,
> Val
>
