Gentle reminder.
+ users.
On Mon, Mar 16, 2020 at 11:27 PM mandeep gandhi <welcomemand...@gmail.com>
wrote:
> Hi,
>
> (PS - I did not use users list as this concerns some internals)
>
> I was trying to deploy the following config on a K8s cluster -
>
> Namespace - X1 Connect group id - G1 Bootstrap servers - B1, B2
> Namespace - X2 Connect group id - G1 Bootstrap servers - B3, B4
>
> With this configuration, I am seeing multiple rebalances (logs below). So
> the question is how is the Membership protocol working?
> I have tried to follow some KIPs and they mostly say that the Bootstrap
> server is the coordinator of the group. If yes, then logically this
> configuration should work just fine as both configs have different
> bootstrap servers. But as far as I have tried to understand the code (and
> tried to run Kafka Connect), it seems like the members get added in the
> group one by one and the head of the list becomes the group co-ordinator. (
> if I change Connect Group id in 2nd config, things work)
>
> Also, I wanted some pointers on what is happening at the server-side vs
> client-side during the protocol.
>
>
> Kindly help.
>
> Logs -
>
> [2020-03-16 10:27:13,386] INFO [Worker clientId=connect-1,
> groupId=streaming-connect] Current config state offset 12 does not match
> group assignment 9164. Forcing rebalance.
> (org.apache.kafka.connect.runtime.distributed.DistributedHerder:942)
> [2020-03-16 10:27:13,386] INFO [Worker clientId=connect-1,
> groupId=streaming-connect] Rebalance started
> (org.apache.kafka.connect.runtime.distributed.WorkerCoordinator:213)
> [2020-03-16 10:27:13,386] INFO [Worker clientId=connect-1,
> groupId=streaming-connect] (Re-)joining group
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:505)
> [2020-03-16 10:27:13,388] INFO [Worker clientId=connect-1,
> groupId=streaming-connect] Successfully joined group with generation 18535
> (org.apache.kafka.clients.consumer.internals.AbstractCoordinator:469)
> [2020-03-16 10:27:13,389] INFO [Worker clientId=connect-1,
> groupId=streaming-connect] Joined group at generation 18535 and got
> assignment: Assignment{error=0,
> leader='connect-1-3580fdcc-257e-40e7-8243-f7839021599f',
> leaderUrl='http://10.13.191.32:8083/', offset=9164, connectorIds=[],
> taskIds=[], revokedConnectorIds=[], revokedTaskIds=[], delay=0}
> (org.apache.kafka.connect.runtime.distributed.DistributedHerder:1397)
> [2020-03-16 10:27:13,389] WARN [Worker clientId=connect-1,
> groupId=streaming-connect] Catching up to assignment's config offset.
> (org.apache.kafka.connect.runtime.distributed.DistributedHerder:909)
> [2020-03-16 10:27:13,389] INFO [Worker clientId=connect-1,
> groupId=streaming-connect] Current config state offset 12 is behind group
> assignment 9164, reading to end of config log
> (org.apache.kafka.connect.runtime.distributed.DistributedHerder:970)
>
>
>
> Thanks,
>
> Mandeep Gandhi
>
>
>
>
