Thank you, Yash. That additional documentation helps to further my
understanding.
In case it helps in any way, I am currently setting the
rest.advertised.host.name, and listener properties to a private IP address that
is resolvable within each data center. However, each data center can only
co
Hi Yeikel,
> To clarify, who initiates the step that assigns a
> connector to a specific worker? If this process
> is controlled by the leader, wouldn't it result in a
> failure to assign tasks to workers with whom it
> cannot communicate?
This happens via the group rebalance process where each
Thank you, Yash. Your explanation makes sense
To clarify, who initiates the step that assigns a connector to a specific
worker? If this process is controlled by the leader, wouldn't it result in a
failure to assign tasks to workers with whom it cannot communicate?
Although it is not ideal, it i
Hi Yeikel,
Heartbeats and group coordination in Kafka Connect do occur through Kafka,
but a Kafka Connect cluster where all workers cannot communicate with
each other won't work very well. You'll be able to create / update / delete
connectors by making requests to any workers that can communicate
Thank you, Nikhil.
I did notice that challenge you're describing with the REST updates when I had
more than one worker within the same datacenter.
Luckily, solving that was relatively simple as all my workers can communicate
within the same data center, and all I need to do is to ensure that th
Kudos to Nikhil.
your explanation adds to my knowledge.
🙏
On Mon, 25 Sep 2023 at 7:16 PM, Nikhil Srivastava <
nikhilsrivastava4...@gmail.com> wrote:
> Hi Yeikel,
>
> Sharing my two cents. Would let others chime in to add to this.
>
> Based on my understanding, if connect workers (which are all p
Hi Yeikel,
Sharing my two cents. Would let others chime in to add to this.
Based on my understanding, if connect workers (which are all part of the
same cluster) can communicate with the kafka brokers (which happens to be
the Group Coordinator and facilitates Connect Leader Election via Group
Mem
Hello everyone,I'm currently designing a new Kafka Connect cluster, and I'm
trying to understand how connectivity functions among workers.In my setup, I
have a single Kafka Connect cluster connected to the same Kafka topics and
Kafka cluster. However, the workers are deployed in geographically s
